Restarting exchange federation link downstream node leaves orphan queue on upstream

[lukebakken]

Reproduction steps:

• Create two RabbitMQ environments. I'll call them rmq-us for the upstream, rmq-ds for the downstream.
• rmq-us can be a single node, rmq-ds must be a cluster.
• Set up an exchange in the rmq-us environment to be federated.
• Set up a federation link on rmq-ds to federate the exchange from the previous step. This will create the associated queue from which the downstream consumes.
• In the rmq-ds environment, figure out which node is running the federation link.
• Restart the node running the federation link.
• The upstream queue remains, and a new one is created from which to consume. The "orphaned" queue is still bound to the federated exchange so it will fill with messages.

Test project to reproduce:

https://github.com/lukebakken/docker-rabbitmq-cluster

Steps:

https://github.com/lukebakken/docker-rabbitmq-cluster.git
cd docker-rabbitmq-cluster
make image-base
make image-vesc-1034
make up
make import

At this point, you should be able to browse to localhost:15682 to see the upstream management UI, and localhost:15672 to see the downstream. Make a note of which node is running the federation link. Let's say the node is rmq2-ds.local. Run this command to restart it:

docker compose restart rmq2-ds

You should now see the orphaned queue in the upstream, collecting messages.

[BryanBedard]

We are observing the same issue in our environment.

[luos]

FYI, I think this is happening because of this: https://github.com/rabbitmq/rabbitmq-server/issues/7032