search

purestorage / helm-charts

Pure Storage Helm Charts
Apache License 2.0
47 stars 43 forks source link

[Bug] - Deletion of PSOPlugin resource hangs #173

Closed igitcode closed 4 years ago

igitcode commented 4 years ago

Using the operator-k8s-plugin/install.sh script on OpenShift platform.

When attempting to delete the PSOPlugin/psoplugin-operator the deletion hangs. I believe this is due to the metadata "finalizers" parameter. The finalizers value "uninstall-helm-release" is not a valid object to remove.

Example: oc delete PSOPlugin/psoplugin-operator -n 'pure-k8s-operator-installed-namespace' 'hangs'

Expected Behavior: The PSOPlugin/psoplugin-operator gets deleted.

Version: Pulled from Master on Feb. 13th, 2020

sdodsley commented 4 years ago

I believe that the finalizers issue is not something we can resolve in our plugin. This is a known k8s problem. you have to edit the object, manually delete the finalizer and then re-delete. I think this is noted in our documentation

igitcode commented 4 years ago

Are you saying k8s is injecting the finalizer parameter into the PSOPlugin object metadata? If so, can you elaborate on how this is being done? Via some other annotation/label within the cluster? To me it seems that the PSOPlugin CRD or pso-operator is setting this value.

Also, if this is included in your documentation can you please share a link where this is located?

sdodsley commented 4 years ago

I don't think k8s is injecting anything - it just has problems deleting CRDs. Our documentation on this is here: https://github.com/purestorage/helm-charts/tree/master/operator-k8s-plugin#uninstall. If you search in google for CRDs failing to delete correctly you will find quite a few references...

igitcode commented 4 years ago

That is the documentation I followed to delete, but it doesn't not call out that you have to manually remove the "finalizers" parameter. The CRD is failing to delete because of the "finalizers" tag. I'm certain the Pure operator is adding this parameter.

k8s documentation on how finalizers work: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#finalizers

sdodsley commented 4 years ago

Ah - I thought the command to fix this was there....

kubectl patch crd/psoplugins.purestorage.com -p '{"metadata":{"finalizers":[]}}' --type=merge

This will fix the issue. As I said this is a commonly known error with finalizers in CRDs - it is not specific to our plugin CRD. I'll raise a PR to add this to the documentation

igitcode commented 4 years ago

Sir - I understand how to fix the issue. I know Kubernetes. I was able to fix the issue before you gave me these instructions. What I'm saying is the issue is not with CRDs in general. I believe the issue is with the Pure operator injecting this parameter into this CRD implementation. Kubernetes is functioning as expected per it's documentation on the finalizers: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#finalizers.

I appreciate the PR to fix the documentation in the mean time.

sdodsley commented 4 years ago

For clarification, we do not add any finalizers in our CRDs. This appears to be something k8s adds into the CRD in addition to what we define, hence we have no control over it. What we need to do is fully investigate where it comes from, why it is defined as it is and why this cannot be deleted correctly without the patch. Let me do some more research into this and speak with some of the k8s community.

sdodsley commented 4 years ago

With Helm3 now being supported in OpenShift from 4.3 (tech preview) we will no longer be shipping an Operator to install the CSI driver. Helm chart are the only supported method.

igitcode commented 4 years ago

Since kubernetes does not auto inject the Finalizers property on a CRD, this is one way to resolve the issue.