search

pixie-io / pixie

Instant Kubernetes-Native Application Observability
https://px.dev
Apache License 2.0
5.62k stars 431 forks source link

Helm uninstall leaves px-operator namespace in (stuck) terminating state #1917

Open ddelnano opened 6 months ago

ddelnano commented 6 months ago

Describe the bug The helm install process followed by a helm uninstall does not fully clean up all pixie resources in the v0.1.7 operator release. The OLM project added a csv-cleanup finalizer in v0.27.0 that causes the px-operator to get stuck in a terminating state.

To Reproduce Steps to reproduce the behavior:

  1. Install pixie via the helm install instructions
  2. helm -n pl uninstall pixie
  3. Notice that the px-operator is stuck in a terminating state due to the operators.coreos.com/csv-cleanup finalizer

Expected behavior Helm uninstall cleans up the olm, px-operator and pl namespaces.

Logs I don't have the logs handy, but I will attach them to this issue when I trigger the problem again.

ddelnano commented 6 months ago

As a short term workaround, if OLM is installed prior to using the Pixie helm chart, the issue does not occur. This is because OLM won't be a part of the helm installation, which allows the px-operator finalizers defined within OLM to run successfully on helm uninstall.

I'm evaluating options for addressing this finalizer issue in the meantime.

danlhennessy commented 1 month ago

Can confirm I am experiencing the same issue. Have also found that installing via helm, and when uninstalling, running: px delete against the specific cluster before helm uninstall has also worked without finalizers causing operator namespace to get stuck. But yes not ideal, preferable for helm to manage everything