Best tutorial paper on reproducibility?

[matthew-brett]

Hey @jbpoline - and y'all.

What is the best article / source for the students to read about reproducibility, do you think?

We previously suggested:

• Donoho et al 2009 [DMR+09] (you can skip-read the sections on Sparselab, Symmlab and Spherelab);
• Preeyanon et al 2014 [PPB14];
• Wilson et al 2014 [WAB+14].

See: https://bic-berkeley.github.io/psych-214-fall-2016/day_00.html

Are there other, better references we can point them to, on the general problem of reproducibility?

[jbpoline]

I personally really like Wilson et al, but truly there are no one paper that talk about all aspects of reproducibility. Wilson et al focuses on software / coding aspects, but we should accompany this with data management aspects (eg, data annotation / provenance, I often have students listen to This snafu and statistical best practices (maybe some of the Ioannidis ones)

[matthew-brett]

Thanks! Which Ioannidis ones were you thinking of?

[jbpoline]

sorry for the late reply, I think the Button et al 2013 is probably good for the power aspects, but there might be better ones for more general exposition (including winners curse, file drawer, p-hacking, etc). At the moment, I basically have one per topic. 1.Button, K. S. et al. Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience 14, 365–376 (2013).