Rules 6 & 7

[bdevans]

I just wanted to raise a seeming inconsistency in these two rules where Rule 6 states that software versions should be written in the Dockerfile (rather than a separate e.g. requirements.txt) but Rule 7 states scripts should be mounted in the container (which unlike the software versions, makes them more accessible when the workspace is published). Mounting large data sets makes sense but scripts are very lightweight and it's a somewhat fuzzy distinction between scripts and software dependencies if they are key to the workflow for processing/analysing/plotting the data which we wish to reproduce.

[vsoch]

The (unsaid) distinction I think is for a container intended to be used solely for an interactive environment, for which you'd mount a script at runtime. This is one of the rules I don't agree with 0 I personally think for both cases scripts should be included in containers, and actually the user should err on the side of caution and include all the scripts they need, and only leave out data that is so huge it would be infeasible. Why? Because mounting is not reproducible, and for the most part we are writing about the use case of containers that go along with published work. For where the versions get written, in the Dockerfile or a requirements.txt (or similar) file is fine as long as it's included in the container.

[bdevans]

Yes, I agree. I tend to write an external requirements.txt which is read as part of the docker build process but also available for other types of use. Ultimately, as long as scripts and requirements are under version control in the same repository that goes a long way to making it reproducible but I think it would be better to either include scripts at build time as a (custom) software dependency.

[vsoch]

This was already brought up as a concern too: https://twitter.com/iancal/status/1251294579606896643?s=21

[bdevans]

I hadn't seen that but yes, I agree. If you've ever used sumatra a project management tool, it refuses to let you run uncommitted code as a solution to the problem mentioned in the tweet. In our case, including the script at (the end of) the build process is a simpler and more natural solution (as implied by the tweet).

[nuest]

Just had a chat with @bdevans about this, here's my take:

• :+1: to present options "within Dockerfile" and requirements.txt, and mention advantages of both > no strict rule
• Change Rule 7 to focus on Data only, and preserve only the "scripts + Dockerfile should be in same repo" but move that to the old rule 9, the new rule "Use version control".
  • suggest to try to say in two sentences that you can mount (e.g. if using Jupyter container) or copy (e.g. if running the image only via CLI)

[nuest]

@bdevans #79 only partially covers this issue, right?

[bdevans]

@nuest what did I miss?

[nuest]

@bdevans Never mind, I was confused because of the "old rule 9", but the rule reordering was not part of the PR yet. I assume you'll start another PR for that (good call on going smaller PRs btw).

[bdevans]

Yes that's right. I'll start a new PR as I think it would have been too much to digest in one go otherwise and merging with parallel PRs would have been a nightmare!