[REVIEW]: Paper is not enough: Crowdsourcing the T1 mapping common ground via the ISMRM reproducibility challenge

roboneuro commented 10 months ago

Submitting author: !--author-handle-->@mathieuboudreau@pbellec@pbellec<!--end-reviewers-list-- Reproducible preprint: https://preprint.neurolibre.org/10.55458/neurolibre.00023 Repository archive: 10.5281/zenodo.10589600 Data archive: 10.5281/zenodo.10589602 Book archive: 10.5281/zenodo.10589598 Docker archive: 10.5281/zenodo.10589605

Status

Status badge code:

HTML: <a href="https://neurolibre.org/papers/990f8547642cb2ae8b06be85acd824c3"><img src="https://neurolibre.org/papers/990f8547642cb2ae8b06be85acd824c3/status.svg"></a>
Markdown: [![status](https://neurolibre.org/papers/990f8547642cb2ae8b06be85acd824c3/status.svg)](https://neurolibre.org/papers/990f8547642cb2ae8b06be85acd824c3)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@pbellec, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@roboneuro generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @pbellec know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

@pbellec, please create your checklist typing: @roboneuro generate my checklist

roboneuro commented 10 months ago

Hello humans, I'm @roboneuro, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@roboneuro commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@roboneuro generate pdf

roboneuro commented 10 months ago

Software report:

github.com/AlDanial/cloc v 1.88  T=0.05 s (233.9 files/s, 98796.3 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
TeX                               1             54             12           1599
Jupyter Notebook                  1              0            878           1532
Markdown                          2             80              0            341
YAML                              4             13             17             62
CSS                               1              6              0             46
JSON                              1              0              0              5
Bourne Again Shell                1              0              0              2
--------------------------------------------------------------------------------
SUM:                             11            153            907           3587
--------------------------------------------------------------------------------

gitinspector failed to run statistical information for the repository

roboneuro commented 10 months ago

Wordcount for paper.md is 5322

roboneuro commented 10 months ago

Failed to discover a valid open source license

roboneuro commented 10 months ago

Failed to discover a Statement of need section in paper

roboneuro commented 10 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

roboneuro commented 10 months ago

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- None

MISSING DOIs

- 10.1371/journal.pone.0091318 may be a valid DOI for title: Rapid parametric mapping of the longitudinal relaxation time T1 using two-dimensional variable flip angle magnetic resonance imaging at 1.5 Tesla, 3 Tesla, and 7 Tesla
- 10.1118/1.595535 may be a valid DOI for title: A review of normal tissue hydrogen NMR relaxation times and relaxation mechanisms from 1-100 MHz: dependence on tissue type, NMR frequency, temperature, species, excision, and age
- 10.1186/1532-429x-12-69 may be a valid DOI for title: Shortened Modified Look-Locker Inversion recovery (ShMOLLI) for clinical myocardial T1-mapping at 1.5 and 3 T within a 9 heartbeat breathhold
- 10.1016/j.neuroimage.2021.117744 may be a valid DOI for title: Can MRI measure myelin? Systematic review, qualitative assessment, and meta-analysis of studies validating microstructural imaging with myelin histology
- 10.1088/0031-9155/23/5/012 may be a valid DOI for title: A line scan image study of a tumorous rat leg by NMR
- 10.1016/j.mri.2016.08.021 may be a valid DOI for title: What are normal relaxation times of tissues at 3 T?
- 10.1103/physrev.76.145 may be a valid DOI for title: An Accurate Nuclear Magnetic Resonance Method for Measuring Spin-Lattice Relaxation Times
- 10.1145/2723872.2723882 may be a valid DOI for title: An introduction to Docker for reproducible research
- 10.1002/mrm.1278.abs may be a valid DOI for title: Quantitative imaging of magnetization transfer exchange and relaxation properties in vivo using MRI
- 10.21105/joss.02343 may be a valid DOI for title: qMRLab: Quantitative MRI analysis, under one umbrella
- 10.1109/mcse.2021.3052101 may be a valid DOI for title: Using Jupyter for Reproducible Scientific Workflows
- 10.25080/majora-4af1f417-011 may be a valid DOI for title: Binder 2.0 - Reproducible, interactive, sharable environments for science at scale
- 10.1002/jmri.26598 may be a valid DOI for title: Recommendations towards standards for quantitative MRI (qMRI) and outstanding needs
- 10.1063/1.1684482 may be a valid DOI for title: Time saving in measurement of NMR and EPR relaxation times
- 10.1126/science.171.3976.1151 may be a valid DOI for title: Tumor detection by nuclear magnetic resonance
- 10.1002/mrm.28020 may be a valid DOI for title: Portable and platform-independent MR pulse sequence programs
- 10.1088/0370-1298/62/5/306 may be a valid DOI for title: A Direct Method of Measuring Nuclear Spin-Lattice Relaxation Times
- 10.1186/1532-429x-15-13 may be a valid DOI for title: Normal variation of magnetic resonance T1 relaxation times in the human population at 1.5 T using ShMOLLI
- 10.1016/b978-0-12-817057-1.00004-4 may be a valid DOI for title: Quantitative T1 and T1r Mapping
- 10.1002/jmri.1880070113 may be a valid DOI for title: Modeling tracer kinetics in dynamic Gd-DTPA MR imaging
- 10.1371/journal.pone.0252966 may be a valid DOI for title: Multi-site, multi-platform comparison of MRI T1 measurement using the system phantom
- 10.1002/(sici)1522-2586(199904)9:4<531::aid-jmri4>3.0.co;2-l may be a valid DOI for title: NMR relaxation times in the human brain at 3.0 tesla
- 10.1002/mrm.26235 may be a valid DOI for title: Pulseq: A rapid and hardware-independent pulse sequence prototyping framework
- 10.1016/j.neuroimage.2009.10.002 may be a valid DOI for title: MP2RAGE, a self bias-field corrected sequence for improved segmentation and T1-mapping at high field
- 10.1016/0730-725x(87)90021-x may be a valid DOI for title: Rapid calculation of T1 using variable flip angle gradient refocused imaging

INVALID DOIs

- None

agahkarakuzu commented 10 months ago

@roboneuro set 10.5281/zenodo.841980 as data archive

roboneuro commented 10 months ago

Done, data archive is now 10.5281/zenodo.841980

agahkarakuzu commented 10 months ago

@roboneuro commands

roboneuro commented 10 months ago

Hello @agahkarakuzu, here are the things you can ask me to do:


# List all available commands
@roboneuro commands

# Add to this issue's reviewers list
@roboneuro add @username as reviewer

# Remove from this issue's reviewers list
@roboneuro remove @username from reviewers

# Get a list of all editors's GitHub handles
@roboneuro list editors

# Assign a user as the editor of this submission
@roboneuro assign @username as editor

# Remove the editor assigned to this submission
@roboneuro remove editor

# Remind an author, a reviewer or the editor to return to a review after a 
# certain period of time (supported units days and weeks)
@roboneuro remind @reviewer in 2 weeks

# Check the references of the PDF for missing DOIs
@roboneuro check references

# Perform checks on the repository
@roboneuro check repository

# Adds a checklist for the reviewer using this command
@roboneuro generate my checklist

# Set a value for version
@roboneuro set v1.0.0 as version

# Set a value for branch
@roboneuro set neurolibre-paper as branch

# Set a value for repository
@roboneuro set https://github.com/organization/repo as repository

# Set a value for the archive DOI
@roboneuro set set 10.5281/zenodo.6861996 as archive

# Set a value for the (data/book/repository/docker) archive DOI.
@roboneuro set 10.5281/zenodo.6861996 as data archive

# Set executable book URI for the preprint.
@roboneuro set book uri

# Mention the EiCs for the correct track
@roboneuro ping track-eic

# Reject paper
@roboneuro reject

# Withdraw paper
@roboneuro withdraw

# Invite an editor to edit a submission (sending them an email)
@roboneuro invite @(.*) as editor

# Generates the pdf paper
@roboneuro generate pdf

# Recommends the submission for acceptance
@roboneuro recommend-accept

# Accept and publish the paper
@roboneuro accept

# Update data on an accepted/published paper
@roboneuro reaccept

# Generates a LaTeX preprint file
@roboneuro generate preprint

# Flag submission with questionable scope
@roboneuro query scope

# Get a link to the complete list of reviewers
@roboneuro list reviewers

# Creates a post-review checklist with editor and authors tasks
@roboneuro create post-review checklist

# Open the review issue
@roboneuro start review

# Check the status of the NeuroLibre preview server.
@roboneuro preview server status

# Check the status of the NeuroLibre preprint (production) server.
@roboneuro preprint server status

# Build a NeuroLibre reproducible preprint for technical screening.
@roboneuro build book

# After screening, build book from the forked reository.
@roboneuro production build book

# Transfer data from preview to the preprint (production) server.
@roboneuro production sync data

# Transfer the final preprint (forked repo) to the preprint (production) server (DOI URL).
@roboneuro production sync book

# Start the production process by forking repository and configuring it.
@roboneuro production start

# Creates Zenodo deposits (a.k.a buckets) for archiving reproducibility assets.
@roboneuro zenodo create buckets

# Status check for the archival of the reproducibility assets.
@roboneuro zenodo status

# Copy summary PDF from the preprints repository to the NeuroLibre production server.
@roboneuro production sync pdf

# Upload the latest Jupyter Book to Zenodo for archival.
@roboneuro zenodo upload book

# Upload the latest version of the book repository to Zenodo for archival.
@roboneuro zenodo upload repository

# Upload the latest version of the docker image to Zenodo for archival.
@roboneuro zenodo upload docker

# Upload the submission data to Zenodo for archival. Will not take effect if the data is already archived.
@roboneuro zenodo upload data

# DESTRUCTIVE ACTION: Deletes zenodo records and all the data that has been uploaded. Cannot perform after zenodo publish.
@roboneuro zenodo flush

# Publish reproducibility assets on Zenodo. Sets DOIs if successful.
@roboneuro zenodo publish

# Build an extended PDF for the target repository.
@roboneuro build extended pdf

# Download the submission data.
@roboneuro cache data

agahkarakuzu commented 10 months ago

@roboneuro cache data

agahkarakuzu commented 10 months ago

@roboneuro cache data

agahkarakuzu commented 10 months ago

@roboneuro preview server status

roboneuro commented 10 months ago

🟢 NeuroLibre server is active (running).
🍀 Ready to accept requests from Issue #23
🔗 URL: https://preview.neurolibre.org

agahkarakuzu commented 10 months ago

@roboneuro build book

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success 33e23321 Last updated: 2024-01-25 03:34:14 PST

:information_source: See details

:confetti_ball: Roboneuro will send you the book URL.

roboneuro commented 10 months ago

:confetti_ball::confetti_ball::confetti_ball: Good news!

:hibiscus: Take a loot at the latest version of your NRP

agahkarakuzu commented 10 months ago

@roboneuro build book

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success 6cc9d458 Last updated: 2024-01-25 04:04:38 PST

:information_source: See details

:confetti_ball: Roboneuro will send you the book URL.

roboneuro commented 10 months ago

:confetti_ball::confetti_ball::confetti_ball: Good news!

:hibiscus: Take a loot at the latest version of your NRP

agahkarakuzu commented 10 months ago

@mathieuboudreau can you take a look at the latest build and confirm?

mathieuboudreau commented 10 months ago

Had a quick look @agahkarakuzu - looks good to me!

mathieuboudreau commented 10 months ago

Failed to discover a Statement of need section in paper

Woops my bad - do you want me to add it?

agahkarakuzu commented 10 months ago

@mathieuboudreau no need to add it, that's an automated check that I'll remove later. I'll update the readme md, deal with some references, then should be good to go.

We are going to use the DOI from the previous submission, as data has not changed. Do you confirm?

mathieuboudreau commented 10 months ago

We are going to use the DOI from the previous submission, as data has not changed. Do you confirm?

Actually the data has changed, so please update the DOI for the archived data

agahkarakuzu commented 10 months ago

@roboneuro build book

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success 165b50a4 Last updated: 2024-01-25 11:38:00 PST

:information_source: See details

:confetti_ball: Roboneuro will send you the book URL.

roboneuro commented 10 months ago

:confetti_ball::confetti_ball::confetti_ball: Good news!

:hibiscus: Take a loot at the latest version of your NRP

agahkarakuzu commented 10 months ago

@mathieuboudreau this one is with the updated data, I think it looks good. If you confirm, I'll move on.

agahkarakuzu commented 10 months ago

@roboneuro build book

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success d259b159 Last updated: 2024-01-25 14:59:29 PST

:information_source: See details

:confetti_ball: Roboneuro will send you the book URL.

roboneuro commented 10 months ago

:confetti_ball::confetti_ball::confetti_ball: Good news!

:hibiscus: Take a loot at the latest version of your NRP

agahkarakuzu commented 10 months ago

@roboneuro build extended pdf

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success bfcb30a0 Last updated: 2024-01-25 15:06:25 PST

:information_source: See details

Roboneuro will post a new comment to share the results and provide an explanation of the next steps.

roboneuro commented 10 months ago

📑 Extended PDF has been compiled!

For the submitting author

👀 Please review the extended PDF and verify that all references are accurately included. If everything is correct, please proceed to the next steps. If not, please make the necessary adjustments in the source documents.
⬇️ Download the updated paper.md.
⬇️ Download the updated paper.bib.
ℹ️ Please read and confirm the following:

[!IMPORTANT]
We have added a note in the extended PDF to inform the readers that the narrative content from your notebook content has been automatically added to credit the referenced sources. This note includes citations to the articles explaining the NeuroLibre workflow, integrated research objects, and the Canadian Open Neuroscience Platform (CONP). If you prefer not to include them, please remove the respective citation directives in the updated paper.md before pushing the file to your repository.
- [ ] I, the submitting author, confirm that I have read the note above.
♻️ Update the respective files in your source repository with the files you just downloaded and inform the screener.

For the technical screener

Once the submitting author has updated the repository with the paper.md and paper.bib, please confirm that the PDF successfully builds using the @roboneuro generate pdf.

:warning: However, DO NOT issue @roboneuro build extended pdf command after the submitting author has updated the paper.md and paper.bib.

agahkarakuzu commented 10 months ago

@roboneuro build extended pdf

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success b0327731 Last updated: 2024-01-25 15:12:36 PST

:information_source: See details

Roboneuro will post a new comment to share the results and provide an explanation of the next steps.

roboneuro commented 10 months ago

📑 Extended PDF has been compiled!

For the submitting author

👀 Please review the extended PDF and verify that all references are accurately included. If everything is correct, please proceed to the next steps. If not, please make the necessary adjustments in the source documents.
⬇️ Download the updated paper.md.
⬇️ Download the updated paper.bib.
ℹ️ Please read and confirm the following:

[!IMPORTANT]
We have added a note in the extended PDF to inform the readers that the narrative content from your notebook content has been automatically added to credit the referenced sources. This note includes citations to the articles explaining the NeuroLibre workflow, integrated research objects, and the Canadian Open Neuroscience Platform (CONP). If you prefer not to include them, please remove the respective citation directives in the updated paper.md before pushing the file to your repository.
- [ ] I, the submitting author, confirm that I have read the note above.
♻️ Update the respective files in your source repository with the files you just downloaded and inform the screener.

For the technical screener

Once the submitting author has updated the repository with the paper.md and paper.bib, please confirm that the PDF successfully builds using the @roboneuro generate pdf.

:warning: However, DO NOT issue @roboneuro build extended pdf command after the submitting author has updated the paper.md and paper.bib.

mathieuboudreau commented 10 months ago

@mathieuboudreau this one is with the updated data, I think it looks good. If you confirm, I'll move on.

@agahkarakuzu It actually appears to still be using the old dataset, as there are some very large error bars in the splenium subplot of figure 4.

Using my latest zipped data, here is how it should look:

mathieuboudreau commented 10 months ago

Fixed a bug recently introduced in my original repo while @agahkarakuzu was trying different data_requirements.txt configurations, which lead to the above figure 4 not displaying in the build within that repo. See: https://github.com/rrsg2020/note/issues/1

See current working version of book: https://rrsg2020.github.io/note/

Figure 4 appears the right way like the above comment, not the incorrect way like the latest NRP built by neurolibre: https://preview.neurolibre.org/book-artifacts/rrsg2020/github.com/note/e3a349c78003181502562f5bde1dc525a69ab234/_build/_page/index/singlehtml/

mathieuboudreau commented 10 months ago

@roboneuro build book

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success 4883e7fb Last updated: 2024-01-26 06:14:34 PST

:information_source: See details

:confetti_ball: Roboneuro will send you the book URL.

roboneuro commented 10 months ago

:confetti_ball::confetti_ball::confetti_ball: Good news!

:hibiscus: Take a loot at the latest version of your NRP

mathieuboudreau commented 10 months ago

🎊🎊🎊 Good news!

🌺 Take a loot at the latest version of your NRP

@agahkarakuzu this one looks good to me!

mathieuboudreau commented 10 months ago

@roboneuro build extended pdf

roboneuro commented 10 months ago

🟢 Successfully built

Status: Success deee9e9e Last updated: 2024-01-26 06:46:51 PST

:information_source: See details

Roboneuro will post a new comment to share the results and provide an explanation of the next steps.

roboneuro commented 10 months ago

📑 Extended PDF has been compiled!

For the submitting author

👀 Please review the extended PDF and verify that all references are accurately included. If everything is correct, please proceed to the next steps. If not, please make the necessary adjustments in the source documents.
⬇️ Download the updated paper.md.
⬇️ Download the updated paper.bib.
ℹ️ Please read and confirm the following:

[!IMPORTANT]
We have added a note in the extended PDF to inform the readers that the narrative content from your notebook content has been automatically added to credit the referenced sources. This note includes citations to the articles explaining the NeuroLibre workflow, integrated research objects, and the Canadian Open Neuroscience Platform (CONP). If you prefer not to include them, please remove the respective citation directives in the updated paper.md before pushing the file to your repository.
- [x] I, the submitting author, confirm that I have read the note above.
♻️ Update the respective files in your source repository with the files you just downloaded and inform the screener.

For the technical screener

Once the submitting author has updated the repository with the paper.md and paper.bib, please confirm that the PDF successfully builds using the @roboneuro generate pdf.

:warning: However, DO NOT issue @roboneuro build extended pdf command after the submitting author has updated the paper.md and paper.bib.

mathieuboudreau commented 10 months ago

@roboneuro generate pdf

roboneuro commented 10 months ago

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

neurolibre / neurolibre-reviews