Ability to purge a queue

[jtaubensee]

From @SeanFeldman on December 20, 2016 5:25

There are scenarios where purging a queue is necessary

• Testing scenarios
• Removing all messages because they are no longer needed
• Clearing out deferred messages when sequence numbers are no longer retained by an app

Similar functionality: ASQ`s CloudQueue.ClearAsync().

This would apply to both queue/subscription and associated DLQs.

Copied from original issue: Azure/azure-service-bus-dotnet#46

[EldertGrootenboer]

@max-favilli this issue is for service-side purging. Questions about tools such as Portal Explorer or SBExplorer are better answered on StackOverflow.

Is there any such limitation service wide?

Because even programmatically with receivemessagesasync (https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusreceiver.receivemessagesasync?view=azure-dotnet) no matter how high you set maxMessages and maxWaitTime, if you are receiving from deadletter subqueue, you only get 1 message at time.

There is no limit for this service side. I just tested this, and can receive multiple messages from DLQ as well. The only limitation we have in SB Explorer is if you have enabled large messages on an entity, you can only receive 1 message at a time, as these messages are loaded in the browser process memory. However, if you have large messages enabled on the entity, it will show you a message in SB Explorer indicating this. If this problem persists for you on entities without large messages enabled, please log a support request, including all your namespace information, and we can look into this further through the support process.

[max-favilli]

There is no limit for this service side. I just tested this, and can receive multiple messages from DLQ as well. The only limitation we have in SB Explorer is if you have enabled large messages on an entity, you can only receive 1 message at a time, as these messages are loaded in the browser process memory. However, if you have large messages enabled on the entity, it will show you a message in SB Explorer indicating this. If this problem persists for you on entities without large messages enabled, please log a support request, including all your namespace information, and we can look into this further through the support process.

We do have large messages enabled. And you are right there's a message explaining the behavior in SB Explorer, we didn't notice that.

I am still puzzled why we are not getting multiple messages when trying to receive them programmatically with receivemessagesasync , unfortunatelly we can't currently open a ticket because buying Azure through a partner.

[EldertGrootenboer]

We are currently finalizing work on a feature that will allow batch-delete operations, which will be the first step to a full purge functionality. This batch-delete feature is still planned to release in the first half of 2024.

[palok]

We are currently finalizing work on a feature that will allow batch-delete operations, which will be the first step to a full purge functionality. This batch-delete feature is still planned to release in the first half of 2024.

The first half of 2024 have passed. Any news on this?

I find it confusingly amazing that something so basic remains on the backlog for 6 years. Dont Azure develop it's services with the developer first in mind? I now have to track down some way of emptying my test queue and spend time on that instead of delivering value for my customer.

God I miss AWS.

[EldertGrootenboer]

We have recently released a new batch delete feature, a new feature that lets you delete messages in bulk from your Service Bus entities. Batch delete helps you to delete multiple messages from a messaging entity in one operation, instead of deleting them one by one. Since deletion is done at service side, you don't need to receive the messages before deleting them. For more information, check the announcement and the documentation. We are currently working on support in Service Bus Explorer in the portal.

[SeanFeldman]

Congratulations on the new feature.

Saying that, I don't agree with appropriating this issue for the new feature. Purging a queue is no the same as deleting a batch of messages.

Purging means a a single operation to remove all messages. Batch message delete, accordingly to the documentation:

You can delete a maximum of 4000 messages in a batch delete call. Batch deletion is done on a best-effort basis and doesn’t guarantee the exact messageCount will be deleted in single API call.

For example, with a queue that contains 10K messages, you need at least 3 operations with custom logic to achieve the goal.

I suggest reopening this issue until a true server-side purging is implemented. feature is provided.

[OranguTech]

@SeanFeldman - It's a bit easy to miss, but the docs do say

"Additionally, you can call PurgeMessagesAsync to purge all messages from entity."

Does the PurgeMessagesAsync Method do what we need?

The inline note

"The purge operation could lead to increased CPU usage as it involves multiple API calls."

Is a bit weird, makes wonder if it's just a helper method calling the delete method with some defaults/logic on top or similar.

[SeanFeldman]

Yes and no. Reading the docs, it leaves me thinking this is not a server side purge but rather a client-side driven operation.

This method may invoke multiple service requests to delete all messages. As a result, it may exceed the configured TryTimeout.

And

Because multiple service requests may be made, the possibility of partial success exists. In this scenario, the method will stop attempting to delete additional messages and throw the exception that was encountered. It is recommended to evaluate this exception and determine which messages may not have been deleted.