BAMresearch / Reproducible-Science

Provide ideas, instructions and templates on how to realize reproducible science. https://bamresearch.github.io/Reproducible-Science/
MIT License
0 stars 1 forks source link

Use of term "reproducibility crisis" #3

Closed Psirus closed 4 years ago

Psirus commented 4 years ago

I don't like the phrase "reproducibility crisis in the field of modeling and simulation", because there isn't one in modeling and simulation. The reproducibility crisis a problem where people are unable to reproduce findings in their field. It most strongly affects fields like psychology and behavioural economics, where many of the earlier findings are from studies on a comparatively low number of university students. Now, when researchers tried to replicate these results with larger, more diverse groups, they are unable to do so.

I would argue that the situation in modeling and simulation is decidedly different. While it may be difficult and often needlessly tedious to reproduce the results of others, we have been consistently able to do so (at least in our group). Be it Gradient-Enhanced Damage, porous media flow or phase field models. So while I strongly think we should do more to enable and ease reproducibility, I don't think there is a crisis here.

I would prefer a more nuanced language here for two reasons. One, to keep our efforts in perspective; we are trying to make science more transparent, but might not solve an immediate crisis. Two, to respect the people who are facing an immediate crisis, and be thankful that we don't have to deal with something so complex and irrational as humans :)

TTitscher commented 4 years ago

I totally see your point. That was intended to be very catchy but is indeed not really related to the study. Without too much reformulation/explanations, I suggest avoiding the term crisis and directly start with our idea, like

In the field of modelling and simulation, we can and should ensure reproducibility with a fully automated workflow from the input scripts to a finished document. [...]

Psirus commented 4 years ago

Sounds good. Maybe add a sentence about why we should. From reproducibility.md:

This allows for external verification, greater trust in your results and comparative ease in reconstructing details of your own work in the future.

Something like that