Terminology: reproducibility vs replicability

[ikottlarz]

Hi there, thanks for creating this very nice workshop!

I have a small comment on the "Reproducibility" part of it:

In this section, I got a little confused because the terms "reproducibility" and "replicability" seem to be used as the opposite of what I knew/ was used to. Turns out there's seem to be two "competing" terminologies out there. Maybe it's worth noting this in a future version of the workshop, since there's scientific literature using either?

Literature on the terminology

• Plesser (2018). Reproducibility vs. Replicability: A Brief History of a Confused Terminology: Short article about the confusion in the terminology
• Claerbout and Karrenbach (1992) (this is the definition that is used here (?))
  • Replicability = "writing and then running new software based on the description of a computational model or method provided in the original publication, and obtaining results that are similar enough"
  • Reproducibility = "running the same software on the same input data and obtaining the same results"
• Association for Computing Machinery (2020). Artifact Review and Badging Version 1.1 (definition based on what has been commonly used in the experimental sciences over the last decades)
  • Replicability = "use same code to generate (exactly) the same results"
  • Reproducibility = "use descriptions of algorithms to write own code to reproduce the core results of the study"
• Goodman et al. (2016). What does research reproducibility mean? (new terminology, in attempt to avoid the confusion)
  • Methods reproducibility: provide sufficient detail about procedures and data so that the same procedures could be exactly repeated.
  • Results reproducibility: obtain the same results from an independent study with procedures as closely matched to the original study as possible.
  • Inferential reproducibility: draw the same conclusions from either an independent replication of a study or a reanalysis of the original study.

[Datseris]

Claerbout and Karrenbach (1992) is the definition used here yes.