Receive/Peek only Deferred messages

[abhikt48]

We are receiving bad messages from user and bad messages are turning into Deferred messages. We are trying to find a better solution to handle Deferred messages.

Problem - Queue/Topic have bad messages. Qpid JMS client send "MODIFIED_FAILED_UNDELIVERABLE = 5;" to ServiceBus broker while receiving any bad encoding messages and ServiceBus broker converts these messages to Deferred messages. At this moment Qpid JMS client doesn't have facility to receive Deferred state. Refer - https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-java-how-to-use-jms-api-amqp#amqp-disposition-and-service-bus-operation-mapping

Use Case - Queue/Topic have 2 million good message and 100 bad messages with Deferred state. Need to find a solution to repair only Deferred messages or move to DeadLetter/Any Queue.

Feature Request - Can we have option to receive only Deferred messages from ServiceBus explorer? So we can repair or move to DeadLetter/Any Queue.

It will be highly efficient from support prospective. Please let me know for any query.

[SeanFeldman]

Not possible until the service has support for this feature. @abhikt48 if you'd like this feature please upvote the feature request in the Service Bus issue tracker.

[paolosalvatori]

Thanks @SeanFeldman

[ErikMogensen]

Such a feature would be useful. However, I think it is is possible to implement the requested functionality for this issue using the one of the current SDKs by peeking through the messages to get the sequence numbers for the deferred messages, then use the sequence numbers to receive the messages and finally deadletter them.

[abhikt48]

@SeanFeldman i upvoted your fetaure request.

It will be great if we can plan this feature in upcoming days :)

[SeanFeldman]

then use the sequence numbers to receive the messages and finally deadletter them

Dead-lettering a deferred message is a feature requested from the service team. The public issue is here.

use the sequence numbers to receive the messages

Yeah, about that... https://github.com/Azure/azure-sdk-for-net/issues/15097

Sorry to rain on your parade 🙂

[SeanFeldman]

Disclaimer: this is a very personal comment that is not just for this project but many other OSS projects I've seen experiencing a somewhat similar issue. By no means I'm nitpicking on you @abhikt48 🙂

Regarding

It will be great if we can plan this feature in upcoming days :)

I understand the urgency you have. Please note that the project is based on volunteering personal time, and it is not a paid/financially backed project. If you have an urgency for the feature, you're very welcomed to contribute to the project (it's OSS). I would genuinely refrain from adding urgency to feature requests. @ErikMogensen or @paolosalvatori might be kind enough to donate their personal time and get it in, but I'd not expect that.

[abhikt48]

@SeanFeldman - Thanks for insight. It really helps me to understand situation.

My understanding was wrong- As Microsoft pointed me here - so I thought this project must be backed by them.

Thanks for clarification. I can totally understand. Please take your own time and priorities accordingly.

[ErikMogensen]

Hmm... Looks like https://github.com/Azure/azure-sdk-for-net/issues/15097 is about the normal receive. I meant to use ServiceBusReceiver.ReceiveDeferredMessageAsync().

It seems strange about deadlettering a deferred message, I'll try that and report back here.

[ErikMogensen]

I used the following code and it deadletters a deferred message. It uses Azure.Messaging.ServiceBus. static async Task DeadLetterDeferredMessages(long sequenceNumber) {
await using var client = new ServiceBusClient(connectionString); var receiver = client.CreateReceiver(queueName, new ServiceBusReceiverOptions { ReceiveMode = ServiceBusReceiveMode.PeekLock }); var message = await receiver.ReceiveDeferredMessageAsync(sequenceNumber); await receiver.DeadLetterMessageAsync(message); }

[paolosalvatori]

In the past, I submitted internal feedback several times and asked for the possibility to extend the possibility to receive messages from a queue or subscription by sequence number beyond deferred messages. In fact, this functionality is required to properly implement Message Repair and Resubmit pattern in the Service Bus Explorer tool.

[ErikMogensen]

@abhikt48 , I understand that this is causing you a lot of work but on the other hand I think there are just a few users that would benefit from this functionality so I don't think this will be implemented. Instead I recommend you to create a program based on the sample at https://github.com/Azure/azure-service-bus/tree/master/samples/DotNet/Azure.Messaging.ServiceBus/BasicSendReceiveTutorialwithFilters. Remove the sending and filtering parts and peek instead of receive. When a deferred message is found use the code I showed previously in this thread. I think and hope that will work...

You should run that program periodically to deadletter deferred messages.

[abhikt48]

@ErikMogensen Thanks for input. I am not sure about other organisation, but i can definitely provide your inputs on our organisation. There are more than 500 users within organisation who are doing actively integration with ServiceBus.

We use different apis (Microsoft Java/.Net SDK, Qpid-JMS SDK) for communication with ServiceBus, but we all use ServiceBus explorer to provide support. As ServiceBus explorer is great and handly tool which helps to technical and non technical people as well.

I must request you to please consider this feature which will be grateful for us. I am quite sure this feature will be benificial for other users as well.

[SeanFeldman]

@abhikt48, I don't want to come off as a rude gatekeeper, but I'm intrigued by the following:

There are more than 500 users within organisation who are doing actively integration with ServiceBus.

We use different apis (Microsoft Java/.Net SDK, Qpid-JMS SDK) for communication with ServiceBus, but we all use ServiceBus explorer to provide support. As ServiceBus explorer is great and handly tool which helps to technical and non technical people as well.

I must request you to please consider this feature which will be grateful for us. I am quite sure this feature will be benificial for other users as well.

Do I understand correctly that the organization that uses SBE and considers it a critical tool for its commercial success does not allow one developer to contribute their work time to add a feature to the company's tool is relying upon?

[abhikt48]

@SeanFeldman will raise internally if anyone wish to contribute. I love to contribute but I am from Java background, so I can't.

Will raise this issue with Microsoft as well, if their associate can help us on this.

Will update you if I have any information on this.

[ErikMogensen]

Sounds good @abhikt48.

Unfortunately the latest Service Bus SDKs don't support the State property on messages which would be used to determine if a message is deferred or not. This is a service side issue. There is a feature request for it at https://github.com/Azure/azure-sdk-for-net/issues/10241. That means you would have to resort to the oldest Service Bus SDK for this functionality, Currently 90% of the Service Bus code in Service Bus Explorer uses this library, but we would like to migrate away from it since it blocks new features such as newer versions of .NET from being used. That said I think it will take a long time - if ever - until Service Bus Explorer does not use the old library anymore so probably the Service Bus Product Group will have implemented that feature before then.

[SeanFeldman]

TLDR: there are two types of issue trackers that shouldn't be conflated

1. SDK repo. For .NET SDK repo: https://github.com/Azure/azure-sdk-for-net/issues/
2. Service repo: https://github.com/Azure/azure-service-bus/issues/

Just so that everyone is on the same page -- requesting this feature from the SDK team is not going to yield a lot of results. They depend on the service side to provide the feature, which is raised as an issue for the Service Bus service/broker: https://github.com/Azure/azure-service-bus/issues/95.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.