related work

[mr-c]

https://doi.org/10.12688/f1000research.15140.2

[nuest]

Thank you for the pointer - strongly related.

[psychemedia]

I'd noticed this too, with a mental note to try to compare the points with the rules presented in this repo.

[nuest]

@psychemedia Did you do the comparison yet?

[psychemedia]

Sorry - no; will try to have a look at this tomorrow...

[psychemedia]

So I've done a quick review of the paper, published as Gruening B, Sallou O, Moreno P et al. Recommendations for the packaging and containerizing of bioinformatics software [version 2; peer review: 2 approved, 1 approved with reservations]. F1000Research 2019, 7(ELIXIR):742 (https://doi.org/10.12688/f1000research.15140.2) which has a focus on the containerisation of single applications ("an encapsulated and immutable version of an application, coupled with the bare-minimum operating system components (e.g. dependencies) required for execution") relevant to the bioinformatics domain .

Some things that jumped out wrt current draft of paper drafted in this repo:

• metadata (eg LABEL elements) should include version of containerised application; the container should also be semantically versioned.
• recommends against using ENTRYPOINT (their recommendation 4. Avoid using ENTRYPOINT): Even when the ENTRYPOINT helps the user to get a default behaviour for a tool, it is generally not recommended because of reproducibility concerns of the implicit hidden execution point. By explicitly executing the tool by its executable inside the container (using the container as an environment and not as a fat binary merely through its ENTRYPOINT) the user (e.g. workflow) can recognise and trace the tool that is used within the container.
• (straddles 4. Document within the Dockerfile — Usage instructions and 8. Enable interactive usage and one-click execution) their recommendation 2. One tool, one container recommends that the container containerises a single application; to support use (11. Provide helpful usage message), "If your tool provides a help “-h”, “--help” or “?” message, consider providing this as the default command, “CMD” in case of a Dockerfile."; if no help available, add something to the README.
• (relates to 9. Use a Dockerfile per project): recommendation that as well as sharing containers, Dockerfiles are shared too (10. Provide reproducible and documented builds), along with recommendation 9. Make your package or container discoverable that containers are made discoverable by publishing them to existing bioinformatics registries.
• (relates to 10. Use the container daily) a recommendation to include functional tests (7. Add functional testing logic) over the containerised application somewhere in the container, ideally at a conventional location (eg runTest.sh, available via the internal container PATH).
• regarding image size (which is a consideration because "containers are frequently pushed and pulled (uploaded and downloaded) to/from container registries over the internet, their size matters") 5. Reduce the size of your container as much as possible suggests " the most efficient way [to limit size] is to have two different containers: one for building the app and the second container for deploying the app (which will be the one user will download). While you may need multiple libraries, source code and dependencies for building the app, you should only include the bare necessities in the deployment container, which will actually run the app""; or at the very least, don't install additional "recommended" libraries by default, don't keep build tools in deployed image, do use a lightweight base container".
• there's a handy recommendation 8. Check the license of the software to check software licenses ("The license must always be explicitly defined in your Docker labels") and ensure the license is bundled inside the container if the license requires it. (It strikes that that's the sort of thing that could be part of a build tool - check the license, bundle it if required.)

[psychemedia]

One more, regarding Extension packages and programming language modules, their recommendation 1. A package first, which suggests a) using a package manager for software release in general, b) using Bioconda in particular in their domain:

Bioconda is a channel for the Conda package manager specialised in bioinformatics software. You can create a Conda package by defining a BioConda recipe [example provided]. This recipe includes enough information about the dependencies, the license and fundamental metadata to find, retrieve and use the package. When a recipe is added to Bioconda, it is automatically built into a usable package, tested and made available as a Docker container via Quay.io and as a Singularity container, and is displayed in the BioContainers registry

[nuest]

Thanks @psychemedia for the analysis! Very helpful. I think there is a clear distinction: we worry less about image sizes or single applications, but about packaging whole workflows and making the image the actual research environment.

I tried to connect to the article using your thoughts in https://github.com/nuest/ten-simple-rules-dockerfiles/commit/ee13e359f3a15ead01e3ae3a65bb1bd60eff5fe2

Let me know what you think, please re-open if you think this is not an adequate referencing of the other work.

CC: @vsoch @sje30 @betatim @benmarwick