helm / community

Helm community content
https://helm.sh
421 stars 179 forks source link

Removing crd from the cluster when helmfile apply #341

Open maks1001281 opened 7 months ago

maks1001281 commented 7 months ago

Hi all, I have a problem understanding helm diff and removing crds from kubernetes cluster when helm apply

I have a helmfile which contains 2 charts, the first chart contains crds and other templates and the second one contains crunchy-postgres cluster configuration description, I have clustered these charts in the cluster and everything was working, but there was a need to update and remove crds in a separate chart, for this I released a new tag for the first chart which includes only template files, and released a new chart that includes only the new crds, after applying helmfile -e cpdev diff helm complained that it expects the release-name field with the value crunchy-postgres-operator in the new crds, which was logical since the new chart was now called crunchy-postgres-operator-crds and the current release did not manage it, it was decided to patch the old crd with the command

kubectl patch crds postgresclusters.postgres-operator.crunchydata.com -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator-crds", "meta.helm.sh/release-namespace": "crunchy-postgres"}, "labels":{"app.kubernetes.io/managed-by": "Helm"}}}''

After the patch I saw a normal diff that showed that it removes the old crd and applies the new one, I did helmfile apply and got the crunchy-postgres-operator update, but when updating the operator there were errors and I had to roll back to the old crd and operator version while keeping crunchy-postgres working, I commented out the lines that are associated with the separate chart that only includes crds and changed the tag in the first chart to the old one that includes the template files and the crds files, when applying the helmfile apply I saw that the helmfile expects the old name in the release-name field with the value crunchy-postgres-operator. I patched the new crds to the old fields so that the helmfile would manage it.

kubectl patch crds postgresclusters.postgres-operator.crunchydata.com -p '{"metadata":{"annotations":{"meta.helm. sh/release-name": "crunchy-postgres-operator", "meta.helm.sh/release-namespace": "crunchy-postgres"}, "labels":{"app.kubernetes.io/managed-by": "Helm"}}}''

I applied helmfile diff and saw that old crds are not deleted but only new crds come in, ok I thought, I did helmfile apply and crd was deleted, and since crd deletion deletes all crd related resources my crunchy-postgres cluster with all bases was deleted, why helmfile deleted crd even though it didn't show it in helmfile diff? How helmfile works with crds and why this situation happened?