GoogleCloudPlatform / spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.
Apache License 2.0
423 stars 315 forks source link

spring-cloud-bus-pubsub: subscriptions not deleted / renaming anonymous subscriptions? #2577

Closed Martiul closed 7 months ago

Martiul commented 9 months ago

Describe the bug I have some microservices on GKE configured to be spring-cloud-config clients to listen to configuration file updates (stored in a Git repo) from a spring-cloud-config server. Under the hood, the clients use spring-cloud-gcp-starter-bus-pubsub (5.0.0) (and in turn spring-cloud-stream) to setup an anonymous pubsub subscription to listen to changes from the config server.

However, I noticed that sometimes when service restarts abruptly, the anonymous pubsub is not cleaned up. From the documentation, these subscriptions should be deleted when the connection closes (although, I've noticed that when I turn out the container safety and replace it with a new one, the subscription gets deleted correctly).

As a result, I have a lot of subscriptions in a weird pseudo state. They are still able to send out messages (to the same service) but because since a new deployment creates another subscription, there are now multiple subscriptions to the same single subscriber on the microservice, only one gets acked, while the others raise my alarms about long-standing-unacked pubsub messages.

Basically I have a lot of subscriptions for the same thing, when there should only be one.

I have two question:

  1. Is it a bug for the subscription to not be deleted?
  2. I see in the underlying spring-cloud-stream there is a config to set the subscription name. Following https://stackoverflow.com/questions/40119055/spring-cloud-bus-rename-rabbitmq-queues I've been able to set the name for springCloudBusInput. But this strategy is not documented in https://googlecloudplatform.github.io/spring-cloud-gcp/5.0.1/reference/html/index.html#spring-cloud-bus, and the author of the comment states spring-cloud-bus-pubsub relies on anonymous subscriptions to work properly

Sample NA But I believe deploying a service on GKE that fails abruptly will leave behind a subscription. I had an unhealthy service that would die as soon as it started, and after restarting ~2000 times I was left with ~2000 anonymous subscriptions

burkedavison commented 7 months ago

Resolved through a duplicate internal ticket.

spring-cloud-bus-pubsub is a very thin layer on top of spring-cloud-bus, so we don't think there's a bug in this project to be resolved.

However, we did agree that:

spring.cloud.stream.bindings.springCloudBusInput.group=my-service-subscription