search

Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
586 stars 781 forks source link

Fall Through filter subscription #277

Open mikewo opened 5 years ago

mikewo commented 5 years ago

Description

Actual Behavior

  1. By default if you deliver a message to a topic that doesn't have a subscription, or there are filtered subscriptions that don't end up getting the message because it doesn't match the filters, the message essentially just goes away. No one cared about it, which makes sense for the message to then just disappear.

Expected Behavior

  1. It would be nice to be able to actually create a type of subscription that was to catch these types of messages in order to determine if there is mis-configuration in the system. The goal would be to catch if someone starts sending messages to a topic that no one is paying attention to that type of message, which could be a flag of mis-configuration somewhere. I know there exists the EnableFilteringMessagesBeforePublishing which may be useful in catching these scenarios in development, but I'm also interested in catching them in production as well.
  2. I image this is a special type of Filter you would set when you create the subscription (a peer to Correlation, SQLFilter, etc.) that the server would see and then if a message wasn't delivered to any other subscription a copy of it would be placed in this last subscription.
  3. The system would only allow one of these fall through filters on a topic at a time. Attempting to add another one would result in an exception when attempting to create the subsequent subscription with this type of filter.
SeanFeldman commented 5 years ago

What would you expect to be the behaviour for the messages caught previously when their type is subscribed to later and messages of that type where not consumed from the special subscription? For those messages to remain in this special subscription or get purged?

mikewo commented 5 years ago

They would remain. I don't expect any specific action to occur if someone adds a subscription that starts getting the messages. The "fall through filter" subscription simply wouldn't get any more of them. It would be up to me to know that we've corrected the issue and then perhaps push messages of that type from the fall through filter back through to correctly process.

Ideally, in this set up the fall through filter subscription should be pretty much empty unless something gets misconfigured, or perhaps an existing filtered subscription gets accidentally deleted. Anything landing in this fall through would then show up as big red flag that something is wrong. You'd react to this, correct the issue, and then know you have to reprocess the items, which could mean just having something resubmit the messages back to the topic, and if the system is corrected then they would get processed as expected (aka, they shouldn't end up back in the fall through subscription).

Likewise, if someone turned on a "trueFilter" for a subscription I would expect this fall through filter to basically get nothing, but any existing messages in it would be left alone. Any other approach would mean that if someone turned on a trueFilter, or simply added a subscription with no defined filter (which is just a true filter really), then all the messages in the fall through would then be gone. I don't want that type of behavior. I feel like it's still important for the fall through subscription to be worked and handled as necessary, just as managing a Dead Letter Queue is an exercise that has to happen.

Fabiest commented 5 years ago

I have the same need. Just don't want to lose messages due to some misconfiguration\deletion by mistake.

ig-sinicyn commented 4 years ago

I want to up this issue.

We are migrating from RabbitMq to the ServiceBus and we are stuck with lack of troubleshooting features.

There is no detection of routing failures. The legacy api have TopicDescriptor.EnableFilteringMessagesBeforePublishing feature but the new one does not.

Also, there is no analog of RabbitMQ's alternate exchange, so there's no way to detect routing failures in production.

ig-sinicyn commented 4 years ago

up again:)

EldertGrootenboer commented 2 years ago

We have added this to our backlog to investigate the options around this.

mbrimble commented 1 year ago

cc @EldertGrootenboer I want this. How do i up vote it?

EldertGrootenboer commented 1 year ago

cc @EldertGrootenboer I want this. How do i up vote it?

You can upvote on the original request with a thumbs up, or leave a response like you did. If you have any specific input for this request, please also let us know in the comments here.

cecilphillip commented 1 year ago

Spent a fair amount of time browsing around for solutions until I landed here. It would be great to have an option to set an "unmatched" subscription or queue that these messages would automatically gets forwarded to, then leave if up to the dev to figure out what to do with them.

andyliuliming commented 1 year ago

1000 ups

EldertGrootenboer commented 1 year ago

Thank you for your feedback on this item, it helps us in our efforts to continuously prioritize the different requests we get from all our various feedback channels. We have added this feature in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this feature, as well as explain their scenarios.

EldertGrootenboer commented 6 months ago

This item in our backlog, however we currently don't have an ETA on when development might start on this. For now, to help us give this the right priority, it would be helpful to see others vote and support this item.

Kryptos-FR commented 4 months ago

This is a critical feature. How can we make it happen?

We had some data loss because of a misconfiguration. This feature would have helped us fix the configuration and replay the messages easily. Instead, we had a painful manual process.