search

ReScience / submissions

ReScience C submissions
28 stars 7 forks source link

Ten Years Reproducibility Challenge: paper number #22 #21

Closed agbarnett closed 4 years ago

agbarnett commented 4 years ago

Original article: Barnett AG, Dobson AJ, For the WHO MONICA Project (2004) Estimating trends and seasonality in coronary heart disease. Statist Med 23:3505-3523. DOI: 10.1002/sim.1927

PDF URL: https://github.com/agbarnett/tenyears/raw/master/article/article.pdf Metadata URL: https://github.com/agbarnett/tenyears/raw/master/article/metadata.yaml Code URL: https://github.com/agbarnett/tenyears

Scientific domain: Epidemiology Programming language: SAS Suggested editor:

rougier commented 4 years ago

@agbarnett Thansk for your submission, we'll assign an editor soon. @ctb Can you edit this submission for the Ten Years Challenge (only one reviewer needed)?

rougier commented 4 years ago

Sorry I'm very late. @ctb @benoit-girard @oliviaguest Can any of you edit this submission for the Ten Years Reproducibility Challenge (only 1 reviewer needed). I know it's not in your domain but the https://rescience.github.io/board/ might help.

rougier commented 4 years ago

@ctb @benoit-girard @oliviaguest @khinsen (kind of urgent) Can any of you edit this submission for the Ten Years Reproducibility Challenge (only 1 reviewer needed). I know it's not in your domain but the https://rescience.github.io/board/ might help.

khinsen commented 4 years ago

@rougier I'll try to find a reviewer.

khinsen commented 4 years ago

@arinbasu Can you review this reproducibility report? You may not be able to run the code, which requires proprietary software, but you could still review the article and inspect the code.

khinsen commented 4 years ago

@SergeStinckwich Can you review this reproducibility report in computational epidemiology? Or propose other reviewers?

SergeStinckwich commented 4 years ago

@SergeStinckwich Can you review this reproducibility report in computational epidemiology? Or propose other reviewers?

I don't know SAS but I can help if needed. I will be free after June 28th to do a review.

khinsen commented 4 years ago

@SergeStinckwich Next week is fine! As for SAS, it would be sheer luck to find someone with that competence in the Open Source oriented circle of ReScience. There's a good reason we don't accept proprietary software in regular submissions. We made an exception for this challenge, so we have to find a compromise to deal with it.

SergeStinckwich commented 4 years ago

@khinsen my deadline has been postpone for one week, so I will try to do it this week.

khinsen commented 4 years ago

@SergeStinckwich That would be great, thanks!

agbarnett commented 4 years ago

Any news on this?

rougier commented 4 years ago

@khinsen @SergeStinckwich Gentle reminder

rougier commented 4 years ago

@khinsen Gentle reminder

khinsen commented 4 years ago

@rougier Thanks for the reminder! I got more than 300 GitHub notifications in my inbox a few days ago, covering several weeks, so I must have overlooked quite a few of them :-(

khinsen commented 4 years ago

@SergeStinckwich Are you still available for doing this review?

agbarnett commented 4 years ago

Any news on this?

SergeStinckwich commented 4 years ago

Sorry for the delay. I was able to spend sometimes to read the paper. The paper is well written, the author was able to reproduce the original results. Small differences in the reproduced results are explained clearly. As I don't have access to SAS, I'm not able to reproduce the result from my side. Minor issue: apparently the authors did not put the original code in the git repository.

agbarnett commented 4 years ago

Thanks. The original code is here on github in the 'code' subfolder.

SergeStinckwich commented 4 years ago

Thanks. The original code is here on github in the 'code' subfolder.

In fact, this was not clear for me, what was the difference between original code and new one. Thank you for clarification.

agbarnett commented 4 years ago

I followed the guidelines of uploading the original code first and then any modifications. So you need to look at the file history to see the original and new.

SergeStinckwich commented 4 years ago

Yes I thank you. With only the name of the commits, it was not clear what was the commit corresponding to the original code.

rougier commented 4 years ago

@khinsen Gentle reminder

khinsen commented 4 years ago

@SergeStinckwich Do I understand correctly that in your opinion, this paper can be accepted? We'll have to live with not evaluating the SAS code.

My own impression of the article is that this is a nice (and very readable) story illustrating the importance of properly archiving and documenting code and data.

SergeStinckwich commented 4 years ago

@SergeStinckwich Do I understand correctly that in your opinion, this paper can be accepted? We'll have to live with not evaluating the SAS code.

My own impression of the article is that this is a nice (and very readable) story illustrating the importance of properly archiving and documenting code and data.

Sorry I was not clear enough. Yes I think this paper should be accepted.

khinsen commented 4 years ago

Thanks @SergeStinckwich for the clarification, and of course for the review!

Herewith I proclaim this paper ACCEPTED! I will do the publication steps later today or tomorrow.

khinsen commented 4 years ago

@agbarnett I submitted a pull request with the updated metadata for publication. But I cannot rebuild the article PDF myself because one figure (figures/original_figure.eps) is missing from your repository. Could you please update the PDF yourself using the new metadata file?

agbarnett commented 4 years ago

@khinsen I've added the missing file. Is python the only way to generate the metadata.tex file from metadata.yaml? I've never used python.

khinsen commented 4 years ago

Thanks, with the added file I can rebuild the article myself. The metadata is indeed processed with a Python script, but you don't need to know any Python to use it. All you need is Python 3 plus PyYAML (see https://github.com/rescience/template for instructions).

What I did to rebuild your article is:

I just did another save request at Software Heritage because of the added figure file. I'll then update the SWHID in the metadata, rebuild the article, and submit another pull request with the new stuff. And publish the article on Zenodo of course.

Our software infrastructure is unfortunately still a bit rough. That's the price to pay for getting reproducible article PDFs with permalinks to code. One day mainstream publishing systems will catch up with us!

khinsen commented 4 years ago

The paper is published: http://doi.org/10.5281/zenodo.4290512