search

ReScience / ten-years

Ten Years Reproducibility Challenge
BSD 2-Clause "Simplified" License
62 stars 5 forks source link

Summary of 10 year challenge #15

Open appukuttan-shailesh opened 3 years ago

appukuttan-shailesh commented 3 years ago

I was wondering if an updated summary of the 10 year challenge was posted anywhere?

The only related info I could find was:

As of 24 August 2020, out of 35 researchers who had proposed to reproduce the results of 43 of their old articles, 28 reports had been written, 13 had been accepted after peer review and published, among which 11 documented successful reproductions.

(based on https://www.nature.com/articles/d41586-020-02462-7 and https://en.wikipedia.org/wiki/ReScience_C)

I see here that eventually the publication count rose from 13 to 18.

(hoping to avoid having to manually go through each of them to find the latest stats.

khinsen commented 3 years ago

Wow - I didn't know we have a Wikipedia page!

The original plan was to organize a conference in June 2020 to discuss the challenge and draw conclusions. For well-known reasons, this didn't happen. Many submissions and reviews were delayed as well. I must admit that I neglected the challenge as well over the last months, due to Covid-related overload.

As you can see here, there were 28 submissions in total, out of which ten are still in review. We ought to get these reviews finished before doing a final summary.

appukuttan-shailesh commented 3 years ago

Sure, thats fine. I guess we should have the final numbers in a couple of months or so.

rougier commented 3 years ago

We're a bit late actually...

floe commented 3 years ago

Out of curiosity, are there any plans to re-run the challenge this year?

khinsen commented 3 years ago

Speaking only for myself: I think we should not plan a second round before having reviewed all the submissions of the first one, and drawn some conclusions. One conclusion might be to turn this into a permanent feature, rather than an occasional challenge.

appukuttan-shailesh commented 2 years ago

Do we have any update on this? Are the final numbers now available?

I see here that we still have 5 submissions under review. Two of them seem to be actively worked upon (updates in the last 2-3 months), but not the other three.

rougier commented 2 years ago

I think we can consider the challenge over even if some review are still pending. I second he suggestion of @khinsen of having a new entry of type self-replication with the same conditions (10 years). What do you think @oliviaguest @benoit-girard ?