[REVIEW]: Results of the ISMRM 2020 joint Reproducible Research & Quantitative MR study groups reproducibility challenge on phantom and human brain T1 mapping

[roboneuro]

Submitting author: !--author-handle-->@mathieuboudreau<!--end-author-handle-- (Mathieu Boudreau) Repository: https://github.com/rrsg2020/paper Branch with paper.md (empty if default branch): main Version: v1.0.0 Editor: !--editor-->@pbellec<!--end-editor-- Reviewers: !--reviewers-list-->@pbellec<!--end-reviewers-list-- Reproducible preprint: https://preprint.neurolibre.org/10.55458/neurolibre.00014 Repository archive: 10.5281/zenodo.8419805 Data archive: 10.5281/zenodo.8419809 Book archive: 10.5281/zenodo.8419803 Docker archive: 10.5281/zenodo.8419811

Status

Status badge code:

HTML: <a href="https://neurolibre.org/papers/7d5d631e961003d7770d524747618c7d"><img src="https://neurolibre.org/papers/7d5d631e961003d7770d524747618c7d/status.svg"></a>
Markdown: [![status](https://neurolibre.org/papers/7d5d631e961003d7770d524747618c7d/status.svg)](https://neurolibre.org/papers/7d5d631e961003d7770d524747618c7d)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@pbellec, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review. First of all you need to run this command in a separate comment to create the checklist:

@roboneuro generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @pbellec know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

@pbellec, please create your checklist typing: @roboneuro generate my checklist

[roboneuro]

🔴 Binder build has failed 🥀

---

Status: Failed 74bdc557 Last updated: 2023-07-06 15:28:16 PDT

:information_source: See details

The next comment will forward the logs

[roboneuro]

🥀 We ran into a problem building your book. Please see the log files below.

BinderHub build log

Waiting for build to start...
Picked Git content provider.
Cloning into '/tmp/repo2dockeroq6d3d4f'...
HEAD is now at d27b322 Remove parent zipped folder
Building conda environment for python=3.7Using PythonBuildPack builder
Building conda environment for python=3.7Building conda environment for python=3.7Step 1/53 : FROM buildpack-deps:bionic
 ---> bf89ebf6b3e4
Step 2/53 : ENV DEBIAN_FRONTEND=noninteractive
 ---> Using cache
 ---> ae378de4f5b9
Step 3/53 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends locales > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> 08a78cfb4049
Step 4/53 : RUN echo "en_US.UTF-8 UTF-8" > /etc/locale.gen &&     locale-gen
 ---> Using cache
 ---> fbceb2a67733
Step 5/53 : ENV LC_ALL en_US.UTF-8
 ---> Using cache
 ---> 3cf70612d1fc
Step 6/53 : ENV LANG en_US.UTF-8
 ---> Using cache
 ---> 54727b822f5d
Step 7/53 : ENV LANGUAGE en_US.UTF-8
 ---> Using cache
 ---> dc588c857c02
Step 8/53 : ENV SHELL /bin/bash
 ---> Using cache
 ---> 5e06214b4b44
Step 9/53 : ARG NB_USER
 ---> Using cache
 ---> 42ff2be77138
Step 10/53 : ARG NB_UID
 ---> Using cache
 ---> bd417f3d3dd4
Step 11/53 : ENV USER ${NB_USER}
 ---> Using cache
 ---> b60430c82d10
Step 12/53 : ENV HOME /home/${NB_USER}
 ---> Using cache
 ---> 5e458aff2510
Step 13/53 : RUN groupadd         --gid ${NB_UID}         ${NB_USER} &&     useradd         --comment "Default user"         --create-home         --gid ${NB_UID}         --no-log-init         --shell /bin/bash         --uid ${NB_UID}         ${NB_USER}
 ---> Using cache
 ---> a438632f3dc3
Step 14/53 : RUN wget --quiet -O - https://deb.nodesource.com/gpgkey/nodesource.gpg.key |  apt-key add - &&     DISTRO="bionic" &&     echo "deb https://deb.nodesource.com/node_14.x $DISTRO main" >> /etc/apt/sources.list.d/nodesource.list &&     echo "deb-src https://deb.nodesource.com/node_14.x $DISTRO main" >> /etc/apt/sources.list.d/nodesource.list
 ---> Using cache
 ---> 0942e47ae895
Step 15/53 : RUN apt-get -qq update &&     apt-get -qq install --yes --no-install-recommends        less        nodejs        unzip        > /dev/null &&     apt-get -qq purge &&     apt-get -qq clean &&     rm -rf /var/lib/apt/lists/*
 ---> Using cache
 ---> bad1e96f449e
Step 16/53 : EXPOSE 8888
 ---> Using cache
 ---> 73090ab9c46f
Step 17/53 : ENV APP_BASE /srv
 ---> Using cache
 ---> 3a4d18f1d969
Step 18/53 : ENV NPM_DIR ${APP_BASE}/npm
 ---> Using cache
 ---> 0475c612f284
Step 19/53 : ENV NPM_CONFIG_GLOBALCONFIG ${NPM_DIR}/npmrc
 ---> Using cache
 ---> 4a2d8b0fcc6c
Step 20/53 : ENV CONDA_DIR ${APP_BASE}/conda
 ---> Using cache
 ---> d1a029699546
Step 21/53 : ENV NB_PYTHON_PREFIX ${CONDA_DIR}/envs/notebook
 ---> Using cache
 ---> b14a9e7011ae
Step 22/53 : ENV NB_ENVIRONMENT_FILE /tmp/env/environment.lock
 ---> Using cache
 ---> 7bb964d87d10
Step 23/53 : ENV KERNEL_PYTHON_PREFIX ${NB_PYTHON_PREFIX}
 ---> Using cache
 ---> 56b69ac27d81
Step 24/53 : ENV PATH ${NB_PYTHON_PREFIX}/bin:${CONDA_DIR}/bin:${NPM_DIR}/bin:${PATH}
 ---> Using cache
 ---> 55a865e1b0ca
Step 25/53 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2factivate-2dconda-2esh-391af5 /etc/profile.d/activate-conda.sh
 ---> Using cache
 ---> d82b4d87d277
Step 26/53 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2fenvironment-2epy-2d3-2e7-2elock-4f1154 /tmp/env/environment.lock
 ---> Using cache
 ---> 175f3c212aac
Step 27/53 : COPY --chown=1000:1000 build_script_files/-2fusr-2flib-2fpython3-2e8-2fsite-2dpackages-2frepo2docker-2fbuildpacks-2fconda-2finstall-2dminiforge-2ebash-514214 /tmp/install-miniforge.bash
 ---> Using cache
 ---> c1bcf5d3bbd2
Step 28/53 : RUN mkdir -p ${NPM_DIR} && chown -R ${NB_USER}:${NB_USER} ${NPM_DIR}
 ---> Using cache
 ---> 1c2997e02be2
Step 29/53 : USER ${NB_USER}
 ---> Using cache
 ---> 43da6d61d80f
Step 30/53 : RUN npm config --global set prefix ${NPM_DIR}
 ---> Using cache
 ---> a0eb3d6c7bde
Step 31/53 : USER root
 ---> Using cache
 ---> 3cddb7aebc76
Step 32/53 : RUN TIMEFORMAT='time: %3R' bash -c 'time /tmp/install-miniforge.bash' && rm -rf /tmp/install-miniforge.bash /tmp/env
 ---> Using cache
 ---> 1031221c3d3e
Step 33/53 : ARG REPO_DIR=${HOME}
 ---> Using cache
 ---> 6ce4404a9fa0
Step 34/53 : ENV REPO_DIR ${REPO_DIR}
 ---> Using cache
 ---> 040b57734c85
Step 35/53 : WORKDIR ${REPO_DIR}
 ---> Using cache
 ---> a22c4522f25b
Step 36/53 : RUN chown ${NB_USER}:${NB_USER} ${REPO_DIR}
 ---> Using cache
 ---> 487a0e882c0b
Step 37/53 : ENV PATH ${HOME}/.local/bin:${REPO_DIR}/.local/bin:${PATH}
 ---> Using cache
 ---> 942a7191c6f5
Step 38/53 : ENV CONDA_DEFAULT_ENV ${KERNEL_PYTHON_PREFIX}
 ---> Using cache
 ---> a79741aa8269
Step 39/53 : COPY --chown=1000:1000 src/binder/requirements.txt ${REPO_DIR}/binder/requirements.txt
 ---> Using cache
 ---> 2c63b35c78bb
Step 40/53 : USER ${NB_USER}
 ---> Using cache
 ---> 39f1e375bd28
Step 41/53 : RUN ${KERNEL_PYTHON_PREFIX}/bin/pip install --no-cache-dir -r "binder/requirements.txt"
 ---> Using cache
 ---> db293f1f1a52
Step 42/53 : COPY --chown=1000:1000 src/ ${REPO_DIR}
 ---> 3afb9e4f6446
Step 43/53 : LABEL repo2docker.ref="d27b322c653a1c458a1dfe1f161b7c6721cc4074"
 ---> Running in 3c695291ce61
Removing intermediate container 3c695291ce61
 ---> 24c773bc4bd7
Step 44/53 : LABEL repo2docker.repo="https://github.com/rrsg2020/paper"
 ---> Running in f911a78285e2
Removing intermediate container f911a78285e2
 ---> 31caa2270ee8
Step 45/53 : LABEL repo2docker.version="2021.08.0"
 ---> Running in b05d4d7a8f66
Removing intermediate container b05d4d7a8f66
 ---> 9cf4f3591a32
Step 46/53 : USER ${NB_USER}
 ---> Running in 03027b191ea0
Removing intermediate container 03027b191ea0
 ---> 13aad68637d3
Step 47/53 : RUN chmod +x binder/postBuild
 ---> Running in f62825d09151
Removing intermediate container f62825d09151
 ---> 0e93f3f91f03
Step 48/53 : RUN ./binder/postBuild
 ---> Running in c030c72e1bbb
Cloning into 'content/analysis'...
Removing intermediate container c030c72e1bbb
 ---> 446709e44313
Step 49/53 : ENV PYTHONUNBUFFERED=1
 ---> Running in c98629a640fc
Removing intermediate container c98629a640fc
 ---> 17b10cbf51d9
Step 50/53 : COPY /python3-login /usr/local/bin/python3-login
 ---> d2563ca3b613
Step 51/53 : COPY /repo2docker-entrypoint /usr/local/bin/repo2docker-entrypoint
 ---> 751c8dc474c7
Step 52/53 : ENTRYPOINT ["/usr/local/bin/repo2docker-entrypoint"]
 ---> Running in 6e14acf8115e
Removing intermediate container 6e14acf8115e
 ---> 110e548fbb24
Step 53/53 : CMD ["jupyter", "notebook", "--ip", "0.0.0.0"]
 ---> Running in 6094f5c9ca58
Removing intermediate container 6094f5c9ca58
 ---> bac6d2d322aa
{"aux": {"ID": "sha256:bac6d2d322aa29391be92c316476093537e96ef787eb9bee78481f2f3a98b460"}}Successfully built bac6d2d322aa
Successfully tagged binder-registry.conp.cloud/binder-registry.conp.cloud/binder-rrsg2020-2dpaper-f9dfa7:d27b322c653a1c458a1dfe1f161b7c6721cc4074
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Pushing image
Successfully pushed binder-registry.conp.cloud/binder-registry.conp.cloud/binder-rrsg2020-2dpaper-f9dfa7:d27b322c653a1c458a1dfe1f161b7c6721cc4074Built image, launching...
Launching server...
Server requested
2023-07-06T22:25:57.162382Z [Normal] Successfully assigned binderhub/jupyter-rrsg2020-2dpaper-2dg7ae0rfp to neurolibre-master
2023-07-06T22:26:01Z [Normal] Container image "jupyterhub/k8s-network-tools:1.1.2" already present on machine
2023-07-06T22:26:08Z [Normal] Created container block-cloud-metadata
2023-07-06T22:26:09Z [Normal] Started container block-cloud-metadata
2023-07-06T22:26:11Z [Normal] Pulling image "binder-registry.conp.cloud/binder-registry.conp.cloud/binder-rrsg2020-2dpaper-f9dfa7:d27b322c653a1c458a1dfe1f161b7c6721cc4074"
2023-07-06T22:26:32Z [Normal] Successfully pulled image "binder-registry.conp.cloud/binder-registry.conp.cloud/binder-rrsg2020-2dpaper-f9dfa7:d27b322c653a1c458a1dfe1f161b7c6721cc4074" in 21.501042686s
2023-07-06T22:26:40Z [Normal] Created container notebook
2023-07-06T22:26:42Z [Normal] Started container notebook
Server ready at /jupyter/user/rrsg2020-paper-g7ae0rfp/
server running at https://test.conp.cloud/jupyter/user/rrsg2020-paper-g7ae0rfp/

If the BinderHub build looks OK, please see the Jupyter Book build log(s) below.

Jupyter Book build log

Customized book build: traditional paper layout based on index.ipynb
Running Jupyter-Book v0.13.0
Source Folder: /home/jovyan/content
Config Path: /home/jovyan/content/_config.yml
Output Path: /mnt/books/rrsg2020/github.com/paper/d27b322c653a1c458a1dfe1f161b7c6721cc4074/_build/_page/index/singlehtml
Running Sphinx v4.5.0
Adding copy buttons to code blocks...
Adding copy buttons to code blocks...
making output directory... done
myst v0.15.2: MdParserConfig(renderer='sphinx', commonmark_only=False, enable_extensions=['colon_fence', 'dollarmath', 'linkify', 'substitution', 'tasklist'], dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], url_schemes=['mailto', 'http', 'https'], heading_anchors=None, heading_slug_func=None, html_meta=[], footnote_transition=True, substitutions=[], sub_delimiters=['{', '}'], words_per_minute=200)
MyST-NB: Excluded Paths: set()
updating environment: MyST-NB: Potential docnames to execute: ['index']
executing outdated notebooks... Executing: /home/jovyan/content/index.ipynb
ERROR: Execution Failed: /home/jovyan/content/index.ipynb
done
[new config] 1 added, 0 changed, 0 removed
reading sources... [100%] index
ERROR: Couldn't find cache key for notebook file content/index.ipynb. Outputs will not be inserted.
  Last execution failed with traceback saved in /mnt/books/rrsg2020/github.com/paper/d27b322c653a1c458a1dfe1f161b7c6721cc4074/_build/_page/index/singlehtml/reports/index.log

building [mo]: targets for 0 po files that are out of date
building [singlehtml]: all documents
updating environment: MyST-NB: Potential docnames to execute: []
executing outdated notebooks... Executing: /home/jovyan/content/index.ipynb
done
0 added, 0 changed, 0 removed
ERROR: Execution Failed: /home/jovyan/content/index.ipynb
looking for now-outdated files... 1 found
pickling environment... done
checking consistency... done
preparing documents... done
assembling single document... done
writing... done
writing additional files... done
copying images... [ 50%] images/fig1.png
copying images... [100%] images/figure_3_full.png

copying static files... done
copying extra files... done
dumping object inventory... done
build succeeded, 3 warnings.

The HTML page is in ../../mnt/books/rrsg2020/github.com/paper/d27b322c653a1c458a1dfe1f161b7c6721cc4074/_build/_page/index/singlehtml.
Jupyter-book execution failed!

🐞 After inspecting the logs above, you can interactively debug your notebooks on our BinderHub server.

For guidelines, please see the relevant documentation.

🐞 After inspecting the logs above, you can interactively debug your notebooks on our BinderHub server.

For guidelines, please see the relevant documentation.

[agahkarakuzu]

@roboneuro build book

[roboneuro]

🟢 Successfully built

---

Status: Success 8effc9af Last updated: 2023-07-06 15:43:04 PDT

:information_source: See details

The next comment will forward the logs

[roboneuro]

https://preview.neurolibre.org/book-artifacts/rrsg2020/github.com/paper/562fb392105e48b5da7a4b6bfcf82d7504ea7c71/_build/html/

[agahkarakuzu]

Neeh it is at:

https://preview.neurolibre.org/book-artifacts/rrsg2020/github.com/paper/562fb392105e48b5da7a4b6bfcf82d7504ea7c71/_build/_page/index/singlehtml/

[agahkarakuzu]

@roboneuro build pdf

[roboneuro]

I'm sorry human, I don't understand that. You can see what commands I support by typing:

@roboneuro commands

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[agahkarakuzu]

Ugh, looks like the template is not playing out nicely with the long authors and affiliations list.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@mathieuboudreau Jupyter Book narrative content citations are plaintext in the current index.ipynb.

For one or two references, can you add them to the .bib file, then use {cite:p}citation_key_froim_bib format?

I assume that these references were inserted by paperpile, you can export bib using that.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[mathieuboudreau]

@mathieuboudreau Jupyter Book narrative content citations are plaintext in the current index.ipynb.

For one or two references, can you add them to the .bib file, then use {cite:p}citation_key_froim_bib format?

I assume that these references were inserted by paperpile, you can export bib using that.

I don't fully understand - I thought the required neurlibre bib file was only meant for the "neurolibre document" i.e. the two page thing, why does it matter what my jupyter notebook produces and how I format my references in it?

[mathieuboudreau]

I just want this clarified because I think it would take me long time to get them all formated correctly in a bib file (will jupyter book even append it in my single page document? I doubt it)

[mathieuboudreau]

p.s. I'm away July 15-30th

[agahkarakuzu]

Yes, jupyter book appends references, you can see https://oreoni.github.io as an example.

At this time if you can only change 1 or to references into that format, that would be great.

I have access to the repo, I can do it as well.

I am developing a new future to add those citations into the published pdf for giving credit to each reference.

[mathieuboudreau]

Yes, jupyter book appends references, you can see https://oreoni.github.io/ as an example.

Oh, yes I'm aware that for multi-page books, Jupyter Book appends them. I just always assumed that for single page builds, it wouldn't append anything that's not within that single jupyter notebook I'm building

[mathieuboudreau]

@mathieuboudreau Jupyter Book narrative content citations are plaintext in the current index.ipynb.

For one or two references, can you add them to the .bib file, then use {cite:p}citation_key_froim_bib format?

I assume that these references were inserted by paperpile, you can export bib using that.

Done - let me know when I can revert

[mathieuboudreau]

On my end - as expected, does not work for single page builds.

Citation location (shows up as code)

End of page (no references appended)

[agahkarakuzu]

OK thanks, I'll look into this 👍

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.

[agahkarakuzu]

@roboneuro generate pdf

[roboneuro]

:warning: An error happened when generating the pdf.