rebus-org / Rebus.AzureServiceBus

:bus: Azure Service Bus transport for Rebus
https://mookid.dk/category/rebus
Other
33 stars 20 forks source link

Autodelete for subscriptions? #80

Closed ktjn closed 2 years ago

ktjn commented 2 years ago

ASB supports autodelete for subscriptions. Rebus supports this for queues. Is there a way to enable autodeletes for subscriptions as well?

I can even implement this if I get some hints on how to do it.

mookid8000 commented 2 years ago

How would you expect it to work?

Would you expect the subscription to disappear if the subscriber had not consumed anything from it for a while?

Or would you expect the subscription to disappear if there had not been published anything to its topic for a while?

ktjn commented 2 years ago

Was thinking about this: https://docs.microsoft.com/en-us/dotnet/api/microsoft.servicebus.messaging.subscriptiondescription.autodeleteonidle?view=azure-dotnet

Current workaround is updating this after the subscriptions are setup by rebus.

EDIT: Just for info we use this for services with auto scaling and a random queue with auto delete.

                      var serviceBusConnectionStringProperties = ServiceBusConnectionStringProperties.Parse(GetAzureServiceBusConfiguration(configuration, environment));
                      var client = new ServiceBusAdministrationClient(serviceBusConnectionStringProperties.FullyQualifiedNamespace, new DefaultAzureCredential());
                      foreach (var topic in topics)
                      {
                          var subscriptions = client.GetSubscriptionsAsync(topic);
                          await foreach (var subscription in subscriptions)
                          {
                              if (subscription.SubscriptionName == inputQueueName)
                              {
                                  subscription.AutoDeleteOnIdle = TimeSpan.FromMinutes(5);
                                  await client.UpdateSubscriptionAsync(subscription);
                              }
                          }
                      }
mookid8000 commented 2 years ago

Oh, so you have something that actually works right now?

I've discussed this particular feature with someone recently, and somehow we got the impression that auto-delete on subscriptions would not make sense, because they would be considered idle only when they had not received any events for a while, which is definitely NOT what you want.

Rebus uses subscriptions by configuring them to forward messages to the input queue of the consumer - how does that work together with auto-delete on the queue combined with auto-delete on the subscriptions?

ktjn commented 2 years ago

Hmm... I might have got it wrong. The problem is that we use auto delete on idle for queues, which works fine for dynamic queues. But the subscriptions are left even if the queue is removed.

The logical way would be to set auto delete on the subscriptions itself so it is removed when the queue is gone.

I'll do some research regarding this autodelete on subscription.

mookid8000 commented 2 years ago

I just made a few experiments with ASB, and it turned out, unfortunately, that auto-delete-on-idle cannot be combined with forwarding on subscriptions. When the ForwardTo property is set on the subscription, the AutoDeleteOnIdle timeout is simply ignored and set to TimeSpan.MaxValue.

The test is here: https://github.com/rebus-org/Rebus.AzureServiceBus/blob/master/Rebus.AzureServiceBus.Tests/Checks/CheckWhatHappensWhenSubscriptionHasAutodeleteOnIdleSet.cs#L23

ktjn commented 2 years ago

Hehe. Just came to that conclusion myself.

I think I will just clean subscriptions with no valid queue during startup. Auto delete on queues works just fine.

Thanks for the info.

ktjn commented 2 years ago

For anyone having the same problem:

This check if the queue is still active. If not the subscription is removed.

foreach (var topic in topics) 
{
    var subscriptions = client.GetSubscriptionsAsync(topic);
    await foreach (var subscription in subscriptions)
    {
        if (subscription.SubscriptionName.StartsWith(queuePrefix))
        {
            try
            {
                await client.GetQueueAsync(subscription.SubscriptionName);
                log.LogInformation("Subscription {Name} still active", subscription.SubscriptionName);
            }
            catch (ServiceBusException e) when (e.Reason ==
                                                ServiceBusFailureReason.MessagingEntityNotFound)
            {
                log.LogInformation("Found stale subscription {Name}, deleting...", subscription.SubscriptionName);
                try
                {
                    await client.DeleteSubscriptionAsync(topic, subscription.SubscriptionName);
                }
                catch (Exception ex)
                {
                    log.LogWarning(ex, "Failed to delete subscription, ignoring...");
                }
            }
        }
        else 
        {
            log.LogInformation("Subscription {Name} not matching prefix", subscription.SubscriptionName);
        }
    }
}