search

singularityhub / singularity-hpc

Local filesystem registry for containers (intended for HPC) using Lmod or Environment Modules. Works for users and admins.
https://singularity-hpc.readthedocs.io
Mozilla Public License 2.0
111 stars 26 forks source link

GitLab local registry support #471

Closed nrcfieldsa closed 2 years ago

nrcfieldsa commented 2 years ago

Is it currently supported to use a GitLab repository or container registry on a private network with shpc? Is it possible with a config to update the base repository url?

In an HPC project at our organization it is being target to push and pull singularity containers to a gitlab server, using shpc and environment modules to provide simplified user access to software on the cluster. However some of the containers are to be stored on the private registry; while others can be used directly from public registries.

I am interested in other users opinion if this is a good feature to request or contribute. This project seems to meet a number of sought-after features and efforts are appreciated.

So far, in testing, an attempt is made to update the shpc python files in install path for two variables that seem static defined (cannot find config option: https://singularity-hpc.readthedocs.io/en/latest/getting_started/user-guide.html#id3 ):

$ grep -nRi github_url /usr/local/lib/python3.6/site-packages/singularity_hpc-0.0.24-py3.6.egg/shpc
/usr/local/lib/python3.6/site-packages/singularity_hpc-0.0.24-py3.6.egg/shpc/defaults.py:15:# The GitHub repository with recipes
/usr/local/lib/python3.6/site-packages/singularity_hpc-0.0.24-py3.6.egg/shpc/defaults.py:16:github_url = "https://github.com/singularityhub/singularity-hpc"
..
$ grep -A5 -B2 -n 'url =' /usr/local/lib/python3.6/site-packages/singularity_hpc-0.0.24-py3.6.egg/shpc/main/container.py
78-
79-        # Assemble the artifact url
80:        #url = "https://github.com/%s/releases/download/%s/%s.%s.sif" % (
81:        url =  "https://gitlabxyz.nrc-cnrc.gc.ca/singularity-hpc/%s/%s.%s.sif" % (
82-            repo,
83-            github_tag,
84-            prefix,
85-            container_tag,
86-        )

Config options that exist for the base directory paths are local to the host running shpc, for module_base, container_base and namespace options. Thus, setting these doesn't seem to help with our repository access.

vsoch commented 2 years ago

ah I see it! Okay just pushed a fix.

nrcfieldsa commented 2 years ago

working with latest commit (e5d23f058).

vsoch commented 2 years ago

@nrcfieldsa is this resolved for you?

vsoch commented 2 years ago

I'm going to close this issue and associated PR, no response. If anyone comes to this later and still wants the feature please post.