[Bug]: storage metrics aren't updated after tags removed due to retention policy

[skoppe]

zot version

v2.0.3

Describe the bug

We have set the retention policy to remove tags from certain repos. In the log output I can see the tags are flagged as deleted and on the filesystem I can see increased disk availability.

However, the zot_repo_storage_bytes metric endpoint still reflects the old value. Only after pushing to that specific repo is the metric updated.

I believe this is because the nothing in the GC code ends up calling monitoring.SetStorageUsage(is.metrics, is.rootDir, repo).

To reproduce

{
          "distSpecVersion":"1.0.1",
          "storage":{
            "dedupe": true,
            "gc": true,
            "gcDelay": "1h",
            "gcInterval": "1h",
            "rootDirectory":"/data/zot/",
            "retention": {
              "dryRun": false,
              "delay": "1h",
              "policies": [
                {
                  "repositories": ["**/cache"],
                  "keepTags": [{
                    "mostRecentlyPushedCount": 30,
                    "pushedWithin": "720h"
                  }]
                },
                {
                  "repositories": ["**"],
                  "keepTags": [{
                    "patterns": [".*"]
                  }]
                }
              ]
            }
          },
          "http": {
            "address":"0.0.0.0",
            "port":"5000",
            "realm":"zot",
            "tls": {
              "cert": "/path/to/secrets/tls_crt.pem",
              "key": "/path/to/secrets/tls_key.pem"
            },
            "auth": {
              "htpasswd": {
                "path": "/path/to/htpasswd"
              }
            },
            "accessControl": {
              "metrics": {
                "users": ["metrics"]
              },
              "repositories": {
                "**": {
                  "defaultPolicy": ["read", "create", "update"],
                  "anonymousPolicy": ["read"]
                },
                "example/repo": {
                  "defaultPolicy": ["read", "create", "update"],
                  "anonymousPolicy": ["read", "create", "update", "delete"]
                }
              },
              "adminPolicy": {
                "users": ["admin"],
                "actions": ["read", "create", "update", "delete"]
              }
            }
          },
          "log":{
            "level":"info"
          },
          "extensions": {
            "search": {
              "enable": true
            },
            "sync": {
              "enable": false
            },
            "scrub": {
              "enable": false
            },
            "metrics": {
              "enable": true,
              "prometheus": {
                "path": "/metrics"
              }
            },
            "ui": {
              "enable": true
            }
          }
        }

Expected behavior

To have the zot_repo_storage_bytes metric be updated after images are garbage collected.

Screenshots

No response

Additional context

No response

[rchincha]

@skoppe thanks for trying out zot.

This is likely to be the case even for 2.1.x train.