[Bug]: When pushing an image while being unauthorized, the image metadata is created in the catalog

allanger commented 7 months ago

zot version

v2.0.3

Describe the bug

Our zot is configured to accept only read requests from unauthorized users. We've tested whether images can make to the registry when being pushed by unauthorized users.

We've tried using docker/nerdctl/podman. And it seems like image layers are not uploaded, but image metadata is getting to zot when using nerdctl and docker

So if I push an image using docker or nerdctl I can find a name of the image in the catalog, and when pushing with podman, images seem not to appear.

To reproduce

Configuration

{
    "storage":
    {
        "rootDirectory": "/var/lib/registry",
        "dedupe": true,
        "gc": true,
        "gcDelay": "1h",
        "gcInterval": "24h"
    },
    "http":
    {
        "auth": 
        {
            "htpasswd": 
            {
              "path": "/secret/htpasswd"
            }
        },
        "accessControl": 
        {
            "metrics":
            {
              "users": ["prom"]
            },
            "repositories": 
            {
              "**": 
              {
                  "anonymousPolicy": ["read"],
                  "policies": 
                  [
                      {
                          "users": ["admin"],
                          "actions": 
                          [
                              "read",
                              "create",
                              "update",
                              "delete"
                          ]
                      }
                  ]
              }
            }
        },
        "address": "0.0.0.0",
        "port": "5000"
    },
    "log":
    {
        "level": "debug"
    },
    "extensions": {
      "scrub": {
        "enable": true
      },
      "search": {
        "enable": true,
        "cve": {
            "updateInterval": "2h"
        }
      },
      "metrics": {
        "enable": true,
        "prometheus": {
            "path": "/metrics"
        }
      }
    }
}

Client tool used

podman
nerdctl
docker
1. Seen error

New repository/image appears when browsing through the /v2/_catalog, but it seems to be nothing more than metadata

Expected behavior

I would expect not to see an image name in the catalog, if I wasn't allowed to push it

Screenshots

No response

Additional context

No response

andaaron commented 7 months ago

Hi @allanger can you please provide the zot logs produced when pushing an image using nerdctl/docker?

rchincha commented 7 months ago

@allanger just trying to figure out if docker/nerdctl somehow pushing with creds behind the scenes.

Also, best practices for authZ policy would be to set a adminPolicy, defaultPolicy, anonymousPolicy and then the per-user overrides.

allanger commented 7 months ago

Also, best practices for authZ policy would be to set a adminPolicy, defaultPolicy, anonymousPolicy and then the per-user overrides.

Yep, sure. But the thing is that we don't need users, because we're using the registry as a cache. (I've removed that part from the config because I was more or less certain that it's irrelevant, I hope it is)

We only need to have anonymous read and that's it. Admin exists just in case

Hi @allanger can you please provide the zot logs produced when pushing an image using nerdctl/docker?

I'll send logs tomorrow.

allanger commented 7 months ago

It turned out not to be related to podman/nerdctl. And I can't even reproduce it in 100% of cases.

It seems like if you push an image that is already in zot, but using another tag, there is a chance that it's going to add a new metadata, to registry.

But now I can't reproduce it again, and no image appears. I think that it might be related to our configuration, so I guess I'll close this issue.

If I'm able to find a way to reproduce it again, I'll re-open it with more details.

Regarding logs, I cant find anything strange there too, it's just a mix of 404 when blobs are not found and then 401 when client tries to upload a blob. So it seems alright. I can still attach it though, if you'd like

andaaron commented 7 months ago

@allanger I was asking about log messages since the username and Authorization headers are shown for requests if included by the client.

allanger commented 7 months ago

Ah, alright, here goes a couple of entries that are related to that requests (I've just replaced IPs and hostnames):

{
  "level": "info",
  "module": "http",
  "component": "session",
  "clientIP": "1.1.1.1:57614",
  "method": "POST",
  "path": "/v2/namespace5/big-test-image/blobs/uploads/",
  "statusCode": 401,
  "latency": "0s",
  "bodySize": 266,
  "headers": {
    "Accept-Encoding": [
      "gzip"
    ],
    "Content-Length": [
      "0"
    ],
    "User-Agent": [
      "docker/26.0.0 go/go1.21.8 git-commit/8b79278 kernel/6.6.22-linuxkit os/linux arch/amd64 UpstreamClient(Docker-Client/26.0.0 \\(darwin\\))"
    ],
    "X-Forwarded-For": [
      "2.2.2.2"
    ],
    "X-Forwarded-Host": [
      "zot.someurl.net"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "X-Forwarded-Scheme": [
      "https"
    ],
    "X-Real-Ip": [
      "2.2.2.2"
    ],
    "X-Request-Id": [
      "327ed8a17b64fbb16ceb398c5777b1bb"
    ],
    "X-Scheme": [
      "https"
    ]
  },
  "goroutine": 57861,
  "caller": "zotregistry.dev/zot/pkg/api/session.go:132",
  "time": "2024-04-18T09:04:58.317486178Z",
  "message": "HTTP API"
}
{
  "level": "info",
  "module": "http",
  "component": "session",
  "clientIP": "1.1.1.1:49500",
  "method": "HEAD",
  "path": "/v2/namespace5/big-test-image/blobs/sha256:6e9597c2ecd0a2134345ecfe1c231c891fa92e9c814200f7cc435f50397520c5",
  "statusCode": 404,
  "latency": "0s",
  "bodySize": 405,
  "headers": {
    "User-Agent": [
      "docker/26.0.0 go/go1.21.8 git-commit/8b79278 kernel/6.6.22-linuxkit os/linux arch/amd64 UpstreamClient(Docker-Client/26.0.0 \\(darwin\\))"
    ],
    "X-Forwarded-For": [
      "2.2.2.2"
    ],
    "X-Forwarded-Host": [
      "zot.someurl.net"
    ],
    "X-Forwarded-Port": [
      "443"
    ],
    "X-Forwarded-Proto": [
      "https"
    ],
    "X-Forwarded-Scheme": [
      "https"
    ],
    "X-Real-Ip": [
      "2.2.2.2"
    ],
    "X-Request-Id": [
      "159ebf662291dec4e7850a60d6810f36"
    ],
    "X-Scheme": [
      "https"
    ]
  },
  "goroutine": 57885,
  "caller": "zotregistry.dev/zot/pkg/api/session.go:132",
  "time": "2024-04-18T09:04:58.223866952Z",
  "message": "HTTP API"
}

andaaron commented 7 months ago

It doesn't look like the client is sending credentials. I'm still wondering why is some cases these calls would go though.

project-zot / zot