Can I CODECHECK an article if I am doing a repro study?

[GiulioCentorame]

Hi all, I am currently getting involved in a study assessing the computational reproducibility of several studies in psych science. Since the project fits very well with what's been done with CODECHECK (different focus, but most procedures are probably overlapping), shall I try to start my first CODECHECK with some of the articles I am working on already? Or do you think there might be an issue with it (since it wasn't strictly required by the authors)? Cheers!

[nuest]

Hi @GiulioCentorame! Sorry for the delay in answering!

As of now, we're not looking for any more reproductions of already published papers but focus on integrating CODECHECK into peer review processes. However, of course you are welcome to apply the CODECHECK principles where they are useful to you. Depending on the efforts you spend, it could be worth writing one report for each of your reproductions and depositing that as an individual record.

I think you point out a crucial thing: the authors were not aware of it. So, if you want to capture the state of reproducibility, do you plan to publish "failed" reproductions in the same manner? Do you plan to reach out to original authors?

@sje30 any thoughts?

[GiulioCentorame]

It's alright, don't worry :smiley:

The project I am currently in plans on reaching out with the authors at a later stage, in case we absolutely need some help with reproducing them, and publishing a failure just in case the reproduction hasn't gone through at all (combining author's help and publicly available material). Since I am not the PI, I would probably involve him in the discussion too with your permission to explain everything in detail.

[sje30]

not much to add, beyond "go for it" and agree its probably worth talking to the authors sooner rather than later.