CR instance delete invokes helm delete behind the scene. Sometimes, helm delete can take a long time to finish. In such situations, the mutating webhook times out with the following exception:
kubectl delete -f supabase.yaml Error from server (InternalError): error when deleting "supabase-emin.yaml": Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
The actual helm release does get deleted, but because there was a timeout, the CR instance resource does not get deleted.
One way to address this is to make the CR instance delete asynchronous.
CR instance delete invokes helm delete behind the scene. Sometimes, helm delete can take a long time to finish. In such situations, the mutating webhook times out with the following exception:
kubectl delete -f supabase.yaml Error from server (InternalError): error when deleting "supabase-emin.yaml": Internal error occurred: failed calling webhook "platform-as-code.crd-binding.v1": failed to call webhook: Post "https://crd-hook-service.default.svc:443/mutate?timeout=30s": context deadline exceeded (Client.Timeout exceeded while awaiting headers)The actual helm release does get deleted, but because there was a timeout, the CR instance resource does not get deleted.
One way to address this is to make the CR instance delete asynchronous.
Currently, deleteHelmRelease is a synchronous call. https://github.com/cloud-ark/kubeplus/blob/master/platform-operator/helm-pod/main.go#L334
We should change it to async call (run it as a go co-routine).
cc: @eminalparslan