search

singularityhub / sregistry

server for storage and management of singularity images
https://singularityhub.github.io/sregistry
Mozilla Public License 2.0
103 stars 42 forks source link

sregistry with sif images on Gitlab api/v4 #351

Open thdesy opened 3 years ago

thdesy commented 3 years ago

Hi,

I have attached sregistry to our local gitlab instance and can browse my build artifacts [1]. However, pulling an image file from the artifacts fails with [2]. As far as I see, the problem is, that the clients tries to read a .simg, while I had been building sif's with Singularity 3. I already tried to change the name accordingly [3] but without success. Requesting the file works in principle [4].

Unfortunately, I did not find the API definitions, so that I could try to patch the request.

Cheers and thanks, Thomas

[1]

> ~/.local/bin/sregistry search thomas.hartmann/singularitybuilder
[client|gitlab] [database|sqlite:///${HOME}/.singularity/sregistry.db]
Artifact Browsers (you will need path and job id for pull)
1  job_id   browser
2  14322    https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14322/artifacts/browse/build
3  14319    https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14319/artifacts/browse/build

[2]

> ~/.local/bin/sregistry pull --debug  14322,thomas.hartmann/singularitybuilder
[client|gitlab] [database|sqlite://${HOME}/.singularity/sregistry.db]
Looking for artifact build/Singularity.simg for job name build, 14322
https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14322/artifacts/raw/build/Singularity.simg/?inline=false
ERROR Invalid url or permissions https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14322/artifacts/raw/build/Singularity.simg/?inline=false
ERROR Cannot find ${HOME}/.singularity/shub/library/singularitybuilder/thomas.hartmann/:latest.sif

[3]

> ~/.local/bin/sregistry pull --name Singularity.sif 14322,thomas.hartmann/singularitybuilder 
[client|gitlab] [database|sqlite:///${HOME}/.singularity/sregistry.db]
Looking for artifact build/Singularity.simg for job name build, 14322
https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14322/artifacts/raw/build/Singularity.simg/?inline=false
ERROR Invalid url or permissions https://gitlab.desy.de/thomas.hartmann/singularitybuilder/-/jobs/14322/artifacts/raw/build/Singularity.simg/?inline=false
ERROR Cannot find Singularity.sif

[4] curl -L -s --header "PRIVATE-TOKEN: ${SREGISTRY_GITLAB_TOKEN}" "https://gitlab.desy.de/api/v4/projects/588/jobs/14300/artifacts/build/Singularity.sif" -o /tmp/foo.sif

vsoch commented 3 years ago

Are you using Singularity Registry server, or the sregistry client (which is at https://github.com/singularityhub/sregistry-cli). They are different. You should look at the gitlab section in there and open a PR if there is a bug. And if [4] works, why not just use that?