How to pull private images from a registry

[victorsndvg]

Hi @vsoch ,

I still trying your great tool :)

First of all I think there is a small but in this line:

https://github.com/singularityhub/sregistry-cli/blob/master/sregistry/main/registry/pull.py#L62

I think you should move the assignment manifest['selfLink'] = url after checking if the manifest is a dictionary.

Then I try to pull a private image (with the SREGISTRY_CLIENT_SECRETS defined) and I'm not able to pull it.

SREGISTRY_CLIENT_SECRETS contains a path to a plain text file containing the full JSON token. Is this right?

I get the following:

$ singularity run --app registry  sregistry-cli pull mso4sc/sregistry-cli
[client|registry] [database|sqlite:////home/sandman/.singularity/sregistry.db]
ERROR Bad request (400). Is this a private container?

If I hardcoded the manifest (obtained from SRegistry API), I can get the image using the pull command.

$ singularity run --app registry -B pull.py:/opt/sregistry-cli/sregistry/main/registry/pull.py sregistry-cli pull mso4sc/sregistry-cli
[client|registry] [database|sqlite:////root/.singularity/sregistry.db]
Progress |===================================| 100.0% 
[container][new]   mso4sc/sregistry-cli:latest
Success! /root/.singularity/shub/  mso4sc/sregistry-cli:latest.simg

Why I cannot get the manifest using the token?

I really don't understand how to pull private images.

[vsoch]

hey @victorsndvg ! Thanks for finding that manifest bug, I fixed it here:

https://github.com/singularityhub/sregistry-cli/commit/136584d09e593af443f97710a11e80b0e647df58

The SREGISTRY_CLIENT_SECRETS is the same file that you would have downloaded from your registry, it is defaulted to be $HOME/.sregistry if you don't set it.

I'm confused about "hardcoding the manifest obtained from sregistry API" and then using Singularity... it looks like you are running a singularity image with sregistry, but binding some local pull.py script with one in the image? Is that for the manifest bug?

[victorsndvg]

Hi @vsoch ,

I tried to pull a private image from SRegistry using the right token and I always get the bad request code 400 ..., I think this means that is not able to retrieve the manifest.

At this point I don't know if I'm doing something wrong ... or I'm misunderstanding something ...

Then, to check if sregistry-cli is able to pull a private image after getting the proper manifest, I harcoded the manifest in the pull.py that I'm binding into the container (avoiding the bad request code 400 ... message) and it worked. Of course this is not the way to go, only a simple check.

Now the questions is ... Why I'm not able to obtain the manifest of an image of a private SRegistry collection?

[vsoch]

hey @victorsndvg ! This is expected behavior given the current implementation - you can only pull public images. Take a look here:

https://github.com/singularityhub/sregistry/blob/master/shub/apps/api/urls/containers.py#L150

You'll notice that if the collection is private, it will just return 400, otherwise the manifest. This is why you are able to get the manifest and then do the pull. So this presents two bugs, and they actually need to be addressed first at the sregistry repository.

1. The get function linked above should check the request for the credential header. If it's present, then even if the collection is private the manifest should be returned. This also means that the request coming from the client (this repository) needs to add the header as well. This will be almost identical to the one generated for push, but for pull.
2. It should not be the case that you can get a manifest for a private image that you don't have this credential for. So this is a bug (also in sregistry).

I can definitely get these fixed up! However I'm going in for surgery early next week, so likely I won't have time until the end of the month.

[vsoch]

okay, just wanted to let you know I started working on this this evening - the integration into the sregistry I think is good, and now I need to figure out the client. I don't have a good ETA because, well, life, but it will be as soon as I can :)

[victorsndvg]

Ok! As always, no rush.

You have other tasks with more priority ;) Lets keep the issue open till it's fixed.

best wishes and Good luck!

[vsoch]

It's your lucky day! Much earlier than anticipated, see https://github.com/singularityhub/sregistry/pull/77

[victorsndvg]

Wow, awesome!

I will check it on Monday ;)

[victorsndvg]

I think we can close this issue too. Reopen if needed

[Trophime]

Hi, I'm having troubles when retreiving an image from a private local sregistry:

sregistry pull hifimagnet/test:mso4sc
[client|registry] [database|sqlite:////home/LNCMI-G/trophime/.singularity/sregistry.db]
Traceback (most recent call last):
  File "/usr/bin/sregistry", line 11, in <module>
    load_entry_point('sregistry==0.0.72', 'console_scripts', 'sregistry')()
  File "/usr/lib/python3/dist-packages/sregistry/client/__init__.py", line 352, in main
    subparser=subparsers[args.command])
  File "/usr/lib/python3/dist-packages/sregistry/client/pull.py", line 53, in main
    save=do_save)
  File "/usr/lib/python3/dist-packages/sregistry/main/registry/pull.py", line 80, in pull
    if manifest.status_code == 403:
AttributeError: 'dict' object has no attribute 'status_code'

If I comment out the lines in pull.py:

# Still denied
 if manifest.status_code == 403:
                    manifest = 403

it works...

[vsoch]

hey @Trophime ! I just pushed a fix to #88 could you give it a try?

[victorsndvg]

Thanks @vsoch , It's working! :)

[salammemphis]

Hi I am having an issue to pull images from my local singularity registry. Its showing following error. any suggestion is highly appreciated.

Failed to get manifest from Shub: Only the default Singularity Hub registry is suported for now

[vsoch]

Can you please provide a full command? I can read dinosaur minds, but not human ones.... :O