search

paolosalvatori / ServiceBusExplorer

The Service Bus Explorer allows users to connect to a Service Bus namespace and administer messaging entities in an easy manner. The tool provides advanced features like import/export functionality or the ability to test topic, queues, subscriptions, relay services, notification hubs and events hubs.
MIT License
1.98k stars 577 forks source link

Purge DLQ is not clearing DLQ when there are many messages #767

Open ththiem opened 5 months ago

ththiem commented 5 months ago

We are trying to use this tool to clean up some DLQ's that have hundreds of thousands of messages and the tool is not handling it as I would expect it to. I am using the latest version downloaded from here, v.5.0.18.

image

image

It is taking a very long time and is only purging a small number of messages from the queue. At this rate it would take many many hours to purge the DLQ. This is a Premium service bus. Server and Receive timeout are set to 60s.

SeanFeldman commented 5 months ago

@ththiem, what's the average message size? Are you using

I suspect this is due to how the functionality is implemented to work around several issues with ASB. To "purge" a queue, SBE has to receive the messages in ReceiveAndDelete mode. It has to account for session vs non-session messages, track the progress, etc. And if you have hundreds of thousands of messages, I can see that it might not be very speedy.

The good news is that the native purge is being worked on and is planned for the first half of 2024 (https://github.com/Azure/azure-service-bus/issues/1). Once the feature is released, SBE will be updated to take advantage of it.

ththiem commented 5 months ago

The messages range in size from 600 B to 20 KB, all session-based messages

SeanFeldman commented 5 months ago

The size is not that bad. Session-based entities - that might be a bit of a problem. Looking at the code, it's only one session at a time that's being purged https://github.com/paolosalvatori/ServiceBusExplorer/blob/f356ffbd2a88f1f0cf03bb9815657a66f080bfa2/src/ServiceBus/Helpers/ServiceBusPurger.cs#L99 And prefetch is set to 10 messages only https://github.com/paolosalvatori/ServiceBusExplorer/blob/f356ffbd2a88f1f0cf03bb9815657a66f080bfa2/src/ServiceBus/Helpers/QueueServiceBusPurger.cs#L44-L54 This would explain why it's not very performant.

I'd say it's worth waiting for the native purge feature. Unless you can't wait and would like to send a PR.

ththiem commented 5 months ago

Alright, will wait for the feature then.

ErikMogensen commented 3 months ago

I agree that adding more sessions, if possible, would speed up the purging. However, I doubt that increasing the prefetch count will have any positive effect.

@ththiem, does it stop purging even though the queue is not empty?

Edit: Added a question, removed unnecessary advice

ErikMogensen commented 3 months ago

I tested purging > one million messages on a Premium queue without issues. image

Could it be that your computer went into sleep mode?

If not, I think there is something wrong with your queue. Try changing a non-critical property such as "Enable Batched Operations" and Update. Then change it back and Update again. That should recreate the backend and with luck you'll get a better one.

If that does not work either, I recommend you to create a case with Microsoft.