Allow anonymous browsing of container distributions

[grzleadams]

It's possible to browse, for example, Pulp file repositories without authenticating by going to <pulp_url>/pulp/content/<repo>, but when we try to do the same for container distributions (i.e., <pulp_url>/pulp/content/<distribution>) we get 403s (since we're unauthenticated in the browser). I assume this is somehow related to pulp_container content guards/access policies but I'm not sure how I can allow unauthenticated browsing of available container images. Any thoughts on how I can enable that workflow?

[lubosmj]

Such repository listing is not enabled at the moment.

However, if you want to list all available repositories in the containers world, you can access the /v2/_catalog endpoint. Note that you will need to access the endpoint with an authorized token.

https://docker-docs.uclv.cu/registry/spec/api/#listing-repositories https://docs.pulpproject.org/pulp_container/workflows/listing-repositories.html

Background

We use live API that implements Docker Registry HTTP API v2 (images are available at, e.g., /v2/repository/manifests/bla) and a custom handler that is tailored just for serving content (concrete artifacts downloadable from, e.g., /pulp/content/manifests/bla). We issue redirects from the live API to /pulp/content/. Therefore, you cannot access/list distributions/repositories in the same way as for other plugins.

[lubosmj]

@grzleadams, can your users access the catalog endpoint I highlighted and list all repositories that are available to them?

Also, the behaviour behind <pulp_url>/pulp/content/ might be subject to change. We may want to revisit the pros and cons of the approach we currently follow.

[grzleadams]

I've just told people to use pulp-cli to list the distributions and they seem okay with that. I've been playing around with the ingress configuration to deal with /pulp/content (people seem not to like it since it's easy to forget) so if I come up with anything useful I'll send it over.

[grzleadams]

@lubosmj I still haven't been able to get /v2/_catalog to work... I know token authentication is the default but does it not fall back to Basic at all? Because I've tried curling with -u, base64 encoding and passing in the Authorization header, etc., and all I ever get is:

{"errors":[{"code":"UNAUTHORIZED","message":"Authentication credentials were not provided.","detail":{}}]}

Also, even with accessing that endpoint, I'm not sure it would give the information I want to see (it would just list the repositories/distributions, not associating the tags with them). Is there a way to get that information? For example, I would just want to see something like:

image1:
  tag1: <sha256>
  tag2: <sha256>
image2:
  tag3: <sha256>

As far as I can tell, there's not an easy way to associate the tags with the image via Pulp CLI. The tags are under pulp container content list and the images are under pulp container distribution list but I have yet to find a way to connect the two. Am I missing something?

[grzleadams]

FWIW I was able to get the tag list via:

pulp show --href $(pulp show --href $(pulp show --href $(pulp container distribution list --name ${image_name} | jq -r '.[] | .repository') | jq -r '.latest_version_href') | jq -r '.content_summary.present."container.tag".href') | jq -r '.results | .[] | .name'

Feels like there's probably a better way, though...

[ipanova]

v2/_catalog endpoint is part of Registry API and you are right it shows just list of repos ( aka pulp distributions) If you want to access through registry endpoints, then v2/repo-name/tags/list per repo will show the tags names. There is not a registry endpoint that would collect all the repos and display its tags

[ipanova]

$ curl -i https://puffy.example.com/v2/_catalog
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 13 May 2024 15:24:28 GMT
Content-Type: application/json
Content-Length: 63
Connection: keep-alive
Vary: Accept
Allow: GET, HEAD, OPTIONS
Docker-Distribution-Api-Version: registry/2.0
X-Registry-Supports-Signatures: 1
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Correlation-ID: 7c47858465504f34b4d60343a0cd0efa
Access-Control-Expose-Headers: Correlation-ID
Strict-Transport-Security: max-age=15768000

{"repositories":["ipanova/azure","rhosp-rhel8/openstack-cron"]}
$ curl -i https://puffy.example.com/v2/ipanova/azure/tags/list
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 13 May 2024 15:24:57 GMT
Content-Type: application/json
Content-Length: 42
Connection: keep-alive
Vary: Accept
Allow: GET, HEAD, OPTIONS
Docker-Distribution-Api-Version: registry/2.0
X-Registry-Supports-Signatures: 1
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Correlation-ID: 101a4c2e8c04496e8be77dd12b12e825
Access-Control-Expose-Headers: Correlation-ID
Strict-Transport-Security: max-age=15768000

{"name":"ipanova/azure","tags":["latest"]}

[grzleadams]

What's the trick to making authentication to that v2 API work? I grabbed the base64-encoded string out of .docker/config.json and put it in -H "Authorization:Bearer" in my curl, and I get Access to the requested resource is not authorized. The provided Bearer token is invalid. But I'm currently doing a push to the registry, so I know it's valid.

[ipanova]

If you have token auth disabled, then auth is not bearer, but basic -H "Authorization:Basic"

[ipanova]

In case you have bearer token auth enabled then you need to use bearer token that you receive while using basic auth in the following way:

[grzleadams]

Token auth is still enabled, but I followed the docs and still get The provided Bearer token is invalid. I'm not quite sure where things are going sideways...

~$ curl -L https://pulp.<domain>/token/?service=pulp.<domain>
~$ curl -L -H "Content-Type: application/json" -H "Authorization: Bearer <token from previous command>" https://pulp.<domain>/v2
{"errors":[{"code":"UNAUTHORIZED","message":"Access to the requested resource is not authorized. The provided Bearer token is invalid.","detail":{}}]}

[ipanova]

1. Try to access endpoint without credentials you will see in the auth header how to get token:

   
   $ curl -i https://puffy.example.com/v2/_catalog
   HTTP/1.1 401 Unauthorized
   Server: nginx
   Date: Mon, 13 May 2024 15:43:52 GMT
   Content-Type: application/json
   Content-Length: 106
   Connection: keep-alive
   WWW-Authenticate: Bearer realm="https://puffy.example.com/token/",service="puffy.example.com",scope="registry:catalog:*"
   Vary: Accept
   Allow: GET, HEAD, OPTIONS
   Docker-Distribution-Api-Version: registry/2.0
   X-Registry-Supports-Signatures: 1
   X-Frame-Options: DENY
   X-Content-Type-Options: nosniff
   Referrer-Policy: same-origin
   Cross-Origin-Opener-Policy: same-origin
   Correlation-ID: ae3951de0bb044289b6830a2ddfe5161
   Access-Control-Expose-Headers: Correlation-ID

{"errors":[{"code":"UNAUTHORIZED","message":"Authentication credentials were not provided.","detail":{}}]}

2. In the auth header you will see the link to the token server you will need to ask the token from. Note the Auth BASIC header that has base64 encoded creds

$ curl -i 'https://puffy.example.com/token/?service=puffy.example.com&scope=registry:catalog:*' -H "Authorization: Basic YWRtaW46cGFzc3dvcmQ=" HTTP/1.1 200 OK Server: nginx Date: Mon, 13 May 2024 15:46:54 GMT Content-Type: application/json Content-Length: 1283 Connection: keep-alive Vary: Accept Allow: GET, HEAD, OPTIONS X-Frame-Options: DENY X-Content-Type-Options: nosniff Referrer-Policy: same-origin Cross-Origin-Opener-Policy: same-origin Correlation-ID: 76e9685aa98845688d40158aecadddfd Access-Control-Expose-Headers: Correlation-ID Strict-Transport-Security: max-age=15768000

{"expires_in":300,"issued_at":"2024-05-13T15:46:54.197350Z","token":"TOKEN","access_token":"TOKEN"}

3. Take the token and use in the api call. Note the Auth BEARER header that contains the token.

$ curl -i https://puffy.example.com/v2/_catalog -H "Authorization: Bearer TOKEN" HTTP/1.1 200 OK Server: nginx Date: Mon, 13 May 2024 15:47:53 GMT Content-Type: application/json Content-Length: 63 Connection: keep-alive Vary: Accept Allow: GET, HEAD, OPTIONS Docker-Distribution-Api-Version: registry/2.0 X-Registry-Supports-Signatures: 1 X-Frame-Options: DENY X-Content-Type-Options: nosniff Referrer-Policy: same-origin Cross-Origin-Opener-Policy: same-origin Correlation-ID: ee9e9be8475e4f0ba201c8045efad6f1 Access-Control-Expose-Headers: Correlation-ID Strict-Transport-Security: max-age=15768000

{"repositories":["ipanova/azure","rhosp-rhel8/openstack-cron"]}

 You can check whether the token is valid by pasting it here https://jwt.io/ You should see proper `access` 

"access": [ { "type": "registry", "name": "catalog", "actions": [ "*" ] }

I know this is not the easiest workflow, but these manual steps are being taken by podman/docker client on behalf of the user.  By 'this' I meant mostly the auth process, podman nor docker do not have a cli command that would have catalog access implemented..

[ipanova]

@grzleadams you're almost there, just missing the SCOPE in the url when requesting the token! scope is really important because this is what you're asking the access to, e.g.

$ curl -i https://puffy.example.com/v2/ipanova/azure/tags/list
HTTP/1.1 401 Unauthorized
Server: nginx
Date: Mon, 13 May 2024 16:00:16 GMT
Content-Type: application/json
Content-Length: 106
Connection: keep-alive
WWW-Authenticate: Bearer realm="https://puffy.example.com/token/",service="puffy.example.com",scope="repository:ipanova/azure:pull"
Vary: Accept
Allow: GET, HEAD, OPTIONS
Docker-Distribution-Api-Version: registry/2.0
X-Registry-Supports-Signatures: 1
X-Frame-Options: DENY
X-Content-Type-Options: nosniff
Referrer-Policy: same-origin
Cross-Origin-Opener-Policy: same-origin
Correlation-ID: 2c458d51c0704196a36a55addfc2db90
Access-Control-Expose-Headers: Correlation-ID

{"errors":[{"code":"UNAUTHORIZED","message":"Authentication credentials were not provided.","detail":{}}]}

See the scope is different. If no scope is provided it means max. access you get is to /v2/ root endpoint.

[grzleadams]

Yep, I have it working now. I didn't notice that the response from the token endpoint included both token and access_token so I blindly copied to the end of the string and picked up more than I meant to. Thanks for your help!

[ipanova]

@grzleadams Great!

[lubosmj]

I have just realized that you can list all tags with skopeo: https://github.com/containers/skopeo/blob/main/docs/skopeo-list-tags.1.md#examples. This might be useful too.

[lubosmj]

I am closing this issue because there is no plan to implement the listing as it is known in other plugins.