proxy not syncing latest upstream versions

[HaveFun83]

We have an odd issue.
After some time of running the proxy is not syncing the latest upstream version and just provide the cached versions. When a manual pod restart is triggered the sync will work for some time again.

Example: grafana chart harbor.log

2024-03-19T00:46:40Z [INFO] [/controller/replication/transfer/image/transfer.go:182]: 
copying grafana.github.io/helm-charts/grafana:[<omit>,7.3.2,7.3.3,7.3.4,7.3.5,7.3.6](source registry)
to helm-oci/grafana:[<omit>,7.3.2,7.3.3,7.3.4,7.3.5,7.3.6](destination registry)...

latest upstream is 7.3.7 This happens several times with different charts now.

Any suggestion what is happen here and how to fix it?

[HaveFun83]

Still an issue. I enabled the debug log but nothing suspicious. Can someone help here?

I dig a bit in the logs and saw that the following event download index only occur once when the proxy pod is restarted

proxy-2024/03/28 14:45:22 download index: https://metallb.github.io/metallb/index.yaml
proxy-2024/03/28 14:45:22 downloading : https://metallb.github.io/metallb/index.yaml

The next time the replication is triggered the index get not downloaded maybe this is a trace

[HaveFun83]

maybe something like https://github.com/container-registry/helm-charts-oci-proxy/issues/3#issuecomment-1727251080

I tried to set the INDEX_CACHE_TTL: 1 but nothing changed looks like the index.yaml is cached forever and never got refreshed from upstream after the first download. @Vad1mo maybe you can help here?

[Vad1mo]

ok, this issue is on our agenda now.

[HaveFun83]

JFI

i did some tests and changed the following parameter

--- a/registry/manifest/charts.go
+++ b/registry/manifest/charts.go
@@ -171,7 +171,7 @@ func (m *Manifests) GetIndex(repoURLPath string) (*repo.IndexFile, error) {
                        // cache error too to avoid external resource exhausting
                        ttl = m.config.IndexErrorCacheTTl
                }
-               m.cache.SetWithTTL(repoURLPath, res, 1000, ttl)
+               m.cache.SetWithTTL(repoURLPath, res, 100000, ttl)
                return res.c, res.err
        }

@@ -239,7 +239,7 @@ func (m *Manifests) getIndexBytes(url string) ([]byte, error) {
                        // cache error too to avoid external resource exhausting
                        ttl = m.config.IndexErrorCacheTTl
                }
-               m.cache.SetWithTTL(url, res, 1000, ttl)
+               m.cache.SetWithTTL(url, res, 100000, ttl)
                return res.c, res.err
        }

now the upstream index.yaml get downloaded every time the replication within harbor is triggered

[HaveFun83]

@Vad1mo @tpoxa i tested this cost change several days now and it works like expected. Should i open an PR for the cost change within the cache or is there any other solution?