knative-extensions / eventing-natss

NATS streaming integration with Knative Eventing.
Apache License 2.0
38 stars 39 forks source link

Deleting the last `NatsJetStreamChannel` in a namespace when namespace-scoped can result in orphaned resources #544

Open dan-j opened 2 months ago

dan-j commented 2 months ago

Describe the bug If you have a NatsJetStreamChannel in a namespace with the eventing.knative.dev/scope: namespace annotation set, a dispatcher is deployed to that namespace. If the channel is then deleted, the controller deletes the dispatcher deployment, but sometimes the dispatcher doesn't have time to delete the underlying stream and/or consumers.

We can't move the stream/consumer management to the controller, because the credentials to the NATS cluster are based on the dispatcher (and we have different credentials in different namespaces).

What we really need to do is have multiple finalizers on the channel, so the actual channel isn't deleted until both reconcilers have time to finalize the resource. Once all finalizers are removed, it will then be deleted and the controller can remove the dispatcher deployment.

Expected behavior Any underlying resources are cleaned up when a channel is deleted.

To Reproduce See above.

Knative release version Latest

Additional context I already have the start of this fix in a big WIP which includes #542 and #543