Bot finding CODECHECK candidates in preprints [incl. automatic detection of auditability]

nuest commented 4 years ago

It would be great to be notified of a preprint that has the potential to be CODECHECKed. Skimming manuscript titles and abstracts, like I regularly do for EarthArXiv, does not really give me that information. A bot/script could download the PDF though and search for keywords around software/code/data. There are a couple of papers who do automatic detection of open data (can dig up) but none that do automatic detection of open methods or auditability as required by CODECHECK. That would be a cool thesis project :-)

If we can easily discover candidates in preprints, we could approach the authors and try to get in the CODECHECK into the review process, growing the number of journals/editors aware of the idea and principles.

Some more ideas:

bot can create issues in a candidates repository, ideally posting DOI and a short excerpt
the detection could have a percentage value for certainty

nuest commented 4 years ago

https://github.com/quest-bih/oddpub/ is for Open Data Detection in publications, maybe we can extent that to Open Software Detection :-)

nuest commented 4 years ago

https://www.nature.com/articles/sdata201930

See section "Using Keywords to Identify Reproducible Articles":

codecheckers / discussion

Bot finding CODECHECK candidates in preprints [incl. automatic detection of auditability] #5