[NEW FEATURE] shapepipe docker image for canfar

[martinkilbinger]

Canfar has a new science portal (jupyter environment), where we can submit jobs. Potentially easier to use than the existing VMs.

See https://www.canfar.net/science-portal/

Seb told me that if we want to have our own session to run ShapePipe, we need to create a docker image to install SP. He gave me this github repo address for help:

https://github.com/opencadc/science-platform/tree/master/containers/session-containers/skaha-notebook

[sfarrens]

Great @martinkilbinger we can discuss this in the dev meeting tomorrow!

[martinkilbinger]

Updating this issue to modify the v1.4 docker image. Here are the version of python and packages, that are currently fixed in the existing conda environment yml file:

• python 3.9.18
• astromatic-psfex==3.21.1
• astromatic-source-extractor==2.25.0
• astropy==5.2
  • matplotlib==3.8.4
  • numba==0.58.1
  • numpy==1.26.4
  • cs_util==0.1
  • mccd==1.2.4
  • python-pysap==0.2.1

[martinkilbinger]

conda_list.txt Full list of package versions if needed.

[cailmdaley]

Thanks, @martinkilbinger. I have now pinned all the python versions in the Dockerfile, using the versions specified in Martin's shapepipe-1 v1.4 environment and falling back to the main shapepipe master environment when no version was specified in the former. In the future I'd like to add a development version of the container that uses more recent package versions where possible.

I decided not to try to pin the apt-get packages, since it seems that Debian only tends to host one package version per operating system version; this means the Dockerfile would break any time a pinned package was updated in the Debian distribution. I don't expect updates to apt-get packages tend to cause reproducibility problems with shapepipe though.

I have also forked Martin's shapepipe-1 repository and set up automatic Docker builds on the fork. It was remarkably easy. The built container lives here for now, and is accessible by clicking on "packages" just below "releases" on the right side of my shapepipe fork. Once we're happy with the container we can merge my fork back into Martin's branch.

To install and test the new version of the container on candide, you can do:

# build writeable "sandbox" container-- 
# will create a shapepipe-1.4 directory that functions like a vm
apptainer build --sandbox shapepipe-1.4 docker:ghcr.io/cailmdaley/shapepipe:v1.4_docker

apptainer shell --writable shapepipe-1.4

cd /app
shapepipe_run -c ./example/config.ini

Per @fabianhervaspeters' request I have installed ipython and jupyerlab in the container. I also found this guide on connecting the container to VS Code---basically you SSH directly into the container and run VS Code from inside the container. We can try to get this working together next week.

Cheers!