Add episode on "Best practices for sharing reproducible containers" (30-45 min)

[bast]

Overview issue: #2

Some ideas for content. Let's collect more and then I will create a skeleton episode:

• where to place the container recipes 
• how to share
• reproducibility: will the container definition work in 3 years?
• building on github actions
• building on gitlab and deploying to dockerhub or similar
• sharing on zenodo
• importance of publishing not only the image but also the definition

[maikenp]

Pointers: https://coderefinery.github.io/hpc-containers/

We have 30-45 minutes. We said we should have exercises during the session? Or was that for self-study? I remember you mentioning that we only then have ca 20 minutes for the teaching part?

We should maybe write up all possible ways to do things that we can think of, and select 1 that we prefer to teach, and consider if we want to list some of the others for self-study/choice.

Where to place container recipes

• Gitlab/Github - other?
• Show that both in quay and dockerhub - you can see the layers as a last resort if the definition files are not available otherwise
• Reproducibilty: best practices
  • Always base your container on specific versions, not "latest" - if you build again you will get a different image which may have conflicts with other software you install
  • Depends on what else you include - files you download from the web for instance that may not be available in the future - installation via dnf/apt for OSes that are no longer supported can give issues - if you want your container to work after x-years - make it as self contained as you possibly can. We should give examples of what this means - so maybe show 1 dockerfile/definition which may work today, and rewrite/show another version that will work in x years?
• Github actions - If we only have 20 minutes, will there be time?

[bast]

As to exercise: I am not sure. Maybe we can offer one of these two:

• 20 min discussion and 20 min demo
• 20 min discussion and 20 min exercise where participants choose the exercise most relevant for them

[bast]

Github Actions building containers: There could be an exercise where somebody copies a template GitHub repo which has everything set-up and they only replace the definition file and let GitHub build the image. It could fit into a 20 min exercise. But I agree that we should not spend time teaching it.

[bast]

We can also go through this to find good points: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008316

[bast]

Here is an empty scaffold: https://coderefinery.github.io/hpc-containers/sharing/

[maikenp]

We can also go through this to find good points: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008316

That is actually super - I think this covers basically everything we need to talk about. So I think that we can use these points and try to cook it down to the basics and see what we want to use more time on, and what we just want to mention briefly.

[simo-tuomisto]

The best practices-page should mention the container documentation mentioned here: https://github.com/coderefinery/hpc-containers/blob/main/content/building_images.rst#adding-documentation-to-your-image

[maikenp]

The best practices-page should mention the container documentation mentioned here: https://github.com/coderefinery/hpc-containers/blob/main/content/building_images.rst#adding-documentation-to-your-image

I will fix that, thanks!

[bast]

The lesson was presented today. Closing.