[PRE REVIEW]: Collaborative Container Modules with Singularity Registry HPC

[whedon]

Submitting author: @vsoch (Vanessa Sochat) Repository: https://github.com/singularityhub/singularity-hpc Version: 0.0.22 Editor: @diehlpk Reviewers: @rcannood, @zbeekman, @ArangoGutierrez Managing EiC: Arfon Smith

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

Author instructions

Thanks for submitting your paper to JOSS @vsoch. Currently, there isn't an JOSS editor assigned to your paper.

@vsoch if you have any suggestions for potential reviewers then please mention them here in this thread (without tagging them with an @). In addition, this list of people have already agreed to review for JOSS and may be suitable for this submission (please start at the bottom of the list).

Editor instructions

The JOSS submission bot @whedon is here to help you find and assign reviewers and start the main review. To find out what @whedon can do for you type:

@whedon commands

[whedon]

Hello human, I'm @whedon, a robot that can help you with some common editorial tasks.

:warning: JOSS reduced service mode :warning:

Due to the challenges of the COVID-19 pandemic, JOSS is currently operating in a "reduced service mode". You can read more about what that means in our blog post.

For a list of things I can do to help you, just type:

@whedon commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@whedon generate pdf

[whedon]

Failed to discover a Statement of need section in paper

[whedon]

Software report (experimental):

github.com/AlDanial/cloc v 1.88  T=0.39 s (738.3 files/s, 26744.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
YAML                           221             62             31           3732
Python                          34            634            639           1853
Markdown                        10            180              0            583
reStructuredText                 8            558            563            523
TeX                              1             39              0            265
make                             1             28              6            143
Bourne Shell                     5             38             15            114
CSS                              2             16              8             76
Tcl/Tk                           2             25             23             76
Lua                              1             19             14             48
SVG                              1              1              0             38
Dockerfile                       1             10              4             33
-------------------------------------------------------------------------------
SUM:                           287           1610           1303           7484
-------------------------------------------------------------------------------

Statistical information for the repository 'b2686a5029a510a91d9e1a53' was
gathered on 2021/04/24.
The following historical commit information, by author, was found:

Author                     Commits    Insertions      Deletions    % of changes
Vanessasaurus                   17          2159           1718           55.70
vsoch                           14          2884            199           44.30

Below are the number of rows from each author that have survived and are still
intact in the current revision:

Author                     Rows      Stability          Age       % in comments
Vanessasaurus              1546           71.6          0.3               13.84
vsoch                      1580           54.8          0.2               18.80

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- journal.pone.0177459 is OK
- 10.1145/2723872.2723882 is OK

MISSING DOIs

- 10.1145/2063348.2063360 may be a valid DOI for title: Best practices for the deployment and management of production HPC clusters
- 10.1007/978-1-4842-3012-1_7 may be a valid DOI for title: The Opinionated Jupyter Stacks
- 10.1007/978-1-4842-3012-1_6 may be a valid DOI for title: Docker Hub
- 10.1371/journal.pcbi.1005209 may be a valid DOI for title: BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

INVALID DOIs

- http://dx.doi.org/10.1155/2015/243180 is INVALID because of 'https://doi.org/' prefix

[vsoch]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[Kevin-Mattheus-Moerman]

@vsoch thanks for this submission. I have a couple of comments/questions on this submission and the paper.

• [ ] JOSS submissions should have a very clear scientific research application. I have to admit this submission is not my domain of expertise but from reading the paper I struggle to find a clear description of the direct scientific research application. Page 3 refers to "Reproducible Science", is this perhaps the core scientific research relevance of this work? It would be best if your first sections directly spell out the relevance of this work to scientific research. After reading the summary and statement of need the reader should clearly/quickly understand how the software impacts scientific research, and why one would feel the need to cite this work when doing research in this domain. I suggest you work on the summary and statement of need section to more clearly reflect the relevance to scientific research. You may also comment here on this to help clarify this to the editors.

• [ ] This paper is rather long. JOSS papers are typically between 250-1000 words. Can you work on shortening the paper?

[vsoch]

Thanks @Kevin-Mattheus-Moerman ! I can definitely shorten the paper, and let's chat about the statement of need first - I am definitely not great at writing these, but creating, managing, and running scientific software is the core of a researcher's work, so the tool is definitely valuable. There is also precedence in JoSS for this exact kind of tool, namely papers that have been published before:

• VCC: A framework for building containerizedreproducible cluster software environments: this is spot on a similar idea, just with a different implementation strategy
• slurmR: A lightweight wrapper for HPC with Slurm: A wrapper for the job manager on HPC, another similar idea (much more simple than what I'm presenting here).

And even a previous paper of mine that is a tool of the same nature, but instead of relying on deploying a docker-compose application, it's all done with technology native to HPC, and more accessible to users (and of course admins) to share and install scientific software:

• Singularity Registry: Open Source Registry for Singularity Images

And you can see that the project has been consistently active since I created it, which is a clear reflection of community need. The tool here, shpc, grew out of both admins and researchers asking for something like this for a few years until I finally made some time to do it recently. So let's chat about how to fix this up. I do think the core of the tool is around reproducibility, and making the management, use, and design of scientific software on HPC much easier for the user. Here is another shot!

---

Portability and reproducibility of complex software stacks is essential for researchers to perform their work. High Performance Computing (HPC) environments add another level of complexity, where possibly conflicting dependencies must co-exist. Although container technologies like Singularity make it possible to "bring your own environment," without any form of central strategy to manage containers, researchers that seek reproducibility via using containers are tasked with managing their own container collection, often not taking care to ensure that a particular digest or version is used. The reproducibility of the work is at risk, as they cannot easily install and use containers, nor can they share their software with others.

Singularity Registry HPC (shpc) is the first of its kind to provide an easy means for a researcher to add their research software for sharing and collaboration with other researchers to an existing collection of over 200 popular scientific libraries. The software installs these containers as environment modules that are easy to use and read documentation for, and exposes aliases for commands in the container that the researcher can add to his or her pipeline without thinking about complex interactions with a container. The simple addition of an entry to the registry maintained by shpc comes down to adding a yaml file, and after doing this, another researcher can easily install the same software, down to the digest, to reproduce the original work.

I'm pinging my co-author @alecbcs for his comments as well! And really sorry that it's too long - that should be easy to fix.

[Kevin-Mattheus-Moerman]

@vsoch @alecbcs thanks for those comments. Could you update the paper? Once you do you can call @whedon generate pdf here to update it.

[vsoch]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[vsoch]

Okay I was able to get it down to around 1000 words, not including the code snippets (and I checked several papers on the current JoSS home and they are of similar length) so I think it's good for another look!

One thing I'm wondering about - I noticed that when I add multiple references, only the first renders, e.g.,:

[@Santana-Perez2015-wo, @Boettiger2014-cz, @Wandell2015-yt]

And I think this used to work? Is there a different means to add multiple references / citations to a block?

[vsoch]

ping @Kevin-Mattheus-Moerman ! Do we want to look for reviewers? Do you need suggestions?

[kyleniemeyer]

@whedon invite @diehlpk as editor

Can you edit this submission?

[whedon]

@diehlpk has been invited to edit this submission.

[kyleniemeyer]

Okay I was able to get it down to around 1000 words, not including the code snippets (and I checked several papers on the current JoSS home and they are of similar length) so I think it's good for another look!

One thing I'm wondering about - I noticed that when I add multiple references, only the first renders, e.g.,:

[@Santana-Perez2015-wo, @Boettiger2014-cz, @Wandell2015-yt]

And I think this used to work? Is there a different means to add multiple references / citations to a block?

FYI, I believe you should use semicolons between multiple references, rather than commas, as per the example in our docs: https://joss.readthedocs.io/en/latest/submitting.html#example-paper-and-bibliography

[vsoch]

Oh thank you! I missed that. Let's try another render.

[vsoch]

@whedon generate pdf

[whedon]

:point_right::page_facing_up: Download article proof :page_facing_up: View article proof on GitHub :page_facing_up: :point_left:

[vsoch]

That worked great!

[diehlpk]

@whedon invite @diehlpk as editor

Can you edit this submission?

Sure thing!

[diehlpk]

@whedon assign @diehlpk as editor

[whedon]

OK, the editor is @diehlpk

[diehlpk]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- journal.pone.0177459 is OK
- 10.1145/2723872.2723882 is OK

MISSING DOIs

- 10.1145/2063348.2063360 may be a valid DOI for title: Best practices for the deployment and management of production HPC clusters
- 10.1007/978-1-4842-3012-1_7 may be a valid DOI for title: The Opinionated Jupyter Stacks
- 10.1007/978-1-4842-3012-1_6 may be a valid DOI for title: Docker Hub
- 10.1371/journal.pcbi.1005209 may be a valid DOI for title: BIDS apps: Improving ease of use, accessibility, and reproducibility of neuroimaging data analysis methods

INVALID DOIs

- http://dx.doi.org/10.1155/2015/243180 is INVALID because of 'https://doi.org/' prefix

@vsoch please check the missing and invalid DOIs and update the paper accordingly.

[diehlpk]

Hey, @parsa or @stevenrbrandt would you be interested to review this paper?

[diehlpk]

Hey, @nuest, @zbeekman, @AustinTSchaffer, @fabianomenegidio, @shrinandj, or @rcannood would you be interested to review this paper?

[vsoch]

@whedon check references

[whedon]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- journal.pone.0177459 is OK
- 10.1145/2063348.2063360 is OK
- 10.1007/978-1-4842-3012-1_7 is OK
- 10.1007/978-1-4842-3012-1_6 is OK
- 10.1371/journal.pcbi.1005209 is OK
- 10.1145/2723872.2723882 is OK
- 10.1155/2015/243180 is OK

MISSING DOIs

- None

INVALID DOIs

- None

[vsoch]

@diehlpk looks like I already did that when whedon originally reported it!

[rcannood]

Sure!

[diehlpk]

@whedon assign @rcannood as reviewer

[whedon]

OK, @rcannood is now a reviewer

[diehlpk]

@diehlpk looks like I already did that when whedon originally reported it!

Thanks. Could you please recommend some potential reviewers?

[vsoch]

Definitely! Where is the Google sheet with the list?

[diehlpk]

Definitely! Where is the Google sheet with the list?

I pinged in the message above all people from the Google sheet having Docker or Singularity mentioned. If you know other folks that would be great.

[vsoch]

oh I didn't know they didn't need to be in the sheet. That's much easier! So since we are crossing HPC and Singularity I'll recommend @ikaneshiro, @jscook2345, @ArangoGutierrez, @yarikoptic, @michaelmoore10, @hartzell, and @alaindomissy.

[zbeekman]

I’m happy to review! Sounds interesting!

[diehlpk]

@whedon add @zbeekman as reviewer

[whedon]

OK, @zbeekman is now a reviewer

[diehlpk]

@whedon start review

[whedon]

OK, I've started the review over in https://github.com/openjournals/joss-reviews/issues/3311.

[ArangoGutierrez]

Hey sorry to be late to the party, I am more than happy to help here

[diehlpk]

@whedon add @ArangoGutierrez as reviewer

[whedon]

OK, @ArangoGutierrez is now a reviewer