Closed ThomasA closed 7 years ago
Yes, the draft is in the repo (you'll have to compile it though). Suggestions/Comments/Corrections welcome.
Page 4 left column, at the end of the sentence saying "Since independent implementation is a major feature of replication work, ReScience does not allow authors to submit replications of their own research, nor the research of close collaborators", I propose to add: ", nor the research of others using their original code".
I'm not sure to get the difference. But the sentence you underlined needs definitely to be changed since we actually allow "self-replication" if the work is sufficiently old (around 10 years old)
What I meant is: no matter if the replicated work is from a collaborator or not, it cannot be accepted by ReScience if the replication is done using the original code (or just changing a few lines of this original code) rather than by performing a re-implementation. It may sound obvious. Using the original code may even fall out of the definition of replication. Nevertheless, I think it's still useful to write explicitly that this is non-eligible for publication in ReScience.
I am not trying to troll here, but even using the same code could be interesting after a few years as your code might rot invisibly due to new versions of dependencies.
I understand your point, and I agree that this would be interesting. Nevertheless, would it deserve publication of a new article in ReScience?
While "rotten" code is a problem, I'm not sure it is part of the ReScience mission. As a matter of fact, ReScience replication are also doomed in the more or less long term.
@ThomasA Diagnosing software collapse in science is definitely worthwhile, but it's very different from the replication work that ReScience is designed for. It's a purely technical problem (no scientific expertise required), and I hope that in the long run we will have an infrastructure for automatically checking for software collapse, using technology similar to continuous integration testing in software development.
The 2017 Manninen et al paper on the reprodusibility of astrocyte models (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5318440/) should probably be cited somwhere in the introduction, but I have at the moment no good suggestion about where and how... So if anyone has an idea...
Maybe simply there:
"Surprisingly, the computational sciences (in the broad sense) and computer sciences (in the strict sense) are no exception (Donoho, Maleki, Rahman, Shahram, & Stodden, 2009; Manninen, Havela, & Linne, 2017) despite the fact they rely heavily on code and data, which should make them immune to the aforementioned problems."
Using the magic doi2bib.org:
@article{Manninen:2017,
doi = {10.3389/fninf.2017.00011},
url = {https://doi.org/10.3389/fninf.2017.00011},
year = {2017},
month = {feb},
publisher = {Frontiers Media {SA}},
volume = {11},
author = {Tiina Manninen and Riikka Havela and Marja-Leena Linne},
title = {Reproducibility and Comparability of Computational Models for Astrocyte Calcium Excitability},
journal = {Frontiers in Neuroinformatics}
}
I assume we get to see the draft before submission? Thanks for the initiative - just let me know if there is anything else I can do to contribute.