Move to a more reliable chart hosting solution

[max-rocket-internet]

We have periodic problems with chartmuseum running on k8s, e.g: https://github.com/deliveryhero/helm-charts/issues/606 https://github.com/deliveryhero/helm-charts/issues/554

GHCR has been suggested as a good option.

[pierluigilenoci]

@max-rocket-internet, you have already some PR ready to fix this: https://github.com/max-rocket-internet/k8s-event-logger/pull/45

[shubhamv1436]

@max-rocket-internet Its still happening. Anything you can do to continuously monitor and automate the fix.

helm repo add deliveryhero https://charts.deliveryhero.io/
Error: looks like "https://charts.deliveryhero.io/" is not a valid chart repository or cannot be reached: failed to fetch https://charts.deliveryhero.io/index.yaml : 504 Gateway Timeout

[msvticket]

Considering that you have Cloudflare configured as CDN in front of the Chartmuseum a quick work around (while working on a long term solution) should be to configure Cloudflare to serve content even though it is stale (i.e. isn't valid according to cache headers).

[bunnybilou]

You might want to consider adding caching on Cloudflare to reduce the need for frequent origin fetches.

I checked the Argo Helm repository, and they have a max-age of 600 seconds configured for caching: https://argoproj.github.io/argo-helm/index.yaml

[max-rocket-internet]

a quick work around (while working on a long term solution) should be to configure Cloudflare to serve content even though it is stale You might want to consider adding caching on Cloudflare to reduce the need for frequent origin fetches.

Good ideas @bunnybilou @msvticket, I've added this rule for now:

@max-rocket-internet, you have already some PR ready to fix this: https://github.com/max-rocket-internet/k8s-event-logger/pull/45

Thanks but this PR is for Github pages, which we won't do.

We will use use GHCR, I just need to find some time to setup up the CI, hopefully I can do it this week or early next, in the mean time the Cloudflare cache rule should work 🙏

[max-rocket-internet]

I would be grateful if everyone could take a look at this PR 😄

[msvticket]

Unfortunately it doesn't work properly; there seem to be some kind of permission problem:

$ helm template oci://ghcr.io/deliveryhero/helm-charts/node-local-dns
Error: GET "https://ghcr.io/v2/deliveryhero/helm-charts/node-local-dns/tags/list": GET "https://ghcr.io/token?scope=repository%3Adeliveryhero%2Fhelm-charts%2Fnode-local-dns%3Apull&service=ghcr.io": unexpected status code 401: unauthorized: authentication required

It is not a general problem with ghcr. An example of another repository that works is oci://ghcr.io/bank-vaults/helm-charts/vault-operator

[msvticket]

Another problem is that the instructions are incomplete:

$ helm install oci://ghcr.io/deliveryhero/helm-charts/node-local-dns
Error: INSTALLATION FAILED: must either provide a name or specify --generate-name

[max-rocket-internet]

@msvticket can you open a PR to fix that?

[max-rocket-internet]

Unfortunately it doesn't work properly; there seem to be some kind of permission problem:

The problem is internal, I need to change the visibility of the packages, this will take me some time 🕒

[msvticket]

@msvticket can you open a PR to fix that?

No, I use helmfile myself, so I would not be able to test the instructions.

[max-rocket-internet]

The problem is internal, I need to change the visibility of the packages, this will take me some time 🕒

Fixed

Another problem is that the instructions are incomplete:

Fixed