Singularity?

[pibion]

We're still working on installing a python environment that uses scikit-learn. It looks like the installation might be failing because the gcc version is pretty old.

We'd like to use the CDMS singularity container, which is distributed on CVMFS.

Would it be possible to install singularity in our image? https://github.com/sylabs/singularity/blob/main/INSTALL.md.

I'd swear we have documentation about how to update what's installed in our image. I know we're trying to keep it small, but singularity would be super useful as CDMS is thinking about switching entirely to singularity as a means to sharing the software environment.

[zonca]

We are already inside a Docker container, it seems like if we run with --privileged mode, it might be possible to run Singularity inside Docker. https://stackoverflow.com/questions/61574793/how-to-use-singularity-in-docker-container I could test running a privileged container: https://www.golinuxcloud.com/kubernetes-privileged-pod-examples/#Example-1_Create_Kubernetes_Privileged_Pod_With_all_Capabilities and see if singularity works there. If so, I can configure JupyterHub to launch privileged single user Jupyter Notebook sessions.

The other way would be to completely drop Docker and use Singularity as containeratization technology for Kubernetes, see https://docs.sylabs.io/guides/cri/1.0/user-guide/k8s.html, however, this is not supported by Kubespray, so this seems extremely complicated.

[pibion]

@zonca I like the idea of working directly with our singularity containers, but I don't want to complicate our system any more than we absolutely have to.

Just being able to run Singularity within a Jupyter instance would be helpful. It would also be great to be able to run notebooks from kernels in a singularity container but I'm not sure if that's possible.

[zonca]

if we can run singularity inside a container then we can also use a kernel from a singularity container

[pibion]

@zonca any news on singularity running in a privileged container?

[zonca]

I was able to run a privileged container inside Kubernetes running singularity commands. next I'll try to use that inside JupyterHub

[zonca]

@pibion working on this I got another idea. What if instead we transform the Singularity image into a Docker image?

https://stackoverflow.com/questions/60451712/how-to-build-docker-image-from-singularity-image

I think it depends on how complicated is the image. But if you can share one of those images, I can give it a try.

[pibion]

Sorry just saw this! @bloer are you already doing this and putting them on Dockerhub?

[pibion]

Okay, finally found the instructions for accessing a CDMS singularity container! https://confluence.slac.stanford.edu/display/CDMS/Using+Containerized+CDMS+Offline+Releases#UsingContainerizedCDMSOfflineReleases-WheretogettheSingularityimagesfrom?

[zonca]

ok, I was able to get the latest container on cdmsdev, putting some references in https://github.com/det-lab/jupyterhub-deploy-kubernetes-jetstream/tree/cdms/singularity

[zonca]

singularity exec $IMG python3 -c "import cdms; print(cdms.__version__)"

0.11

[zonca]

I can use the singularity image to provide a kernel for Jupyter, i.e. Jupyter runs outside of singularity, only the kernel runs through singularity.

see https://github.com/det-lab/jupyterhub-deploy-kubernetes-jetstream/tree/cdms/singularity/example_singularity_kernel

[zonca]

ok, now once we have a deployment, I am going to try to resume the work I had started about having a Docker container that includes a singularity container and see if that works properly.

[zonca]

@pibion following the instructions you provided last year, I see there is a 5-0-0 image at https://gitlab.com/groups/supercdms/-/container_registries/2826304?before=latest, it is a couple of months old, I'll start testing with that one, if you have a pointer to a different image you'd rather use please let me know.

[zonca]

following https://github.com/zonca/zonca.dev/pull/4 I have some new issues using Singularity 4.1.0, it gives error related to setuid I should try again with 3.1 and see if I can solve the problem.