Azure / azure-service-bus

☁️ Azure Service Bus service issue tracking and samples
https://azure.microsoft.com/services/service-bus
MIT License
585 stars 781 forks source link

Rate Limiting on queue subscriber #554

Closed namansukhwani closed 2 years ago

namansukhwani commented 2 years ago

Can we have a feature of rate limiting the processing for the messages in the subscriber? Like I have applied a rate limit 5 messages per sec on my queue receiver, this rate limit should work for all receiver available on multiple deployed pods. Now only total of 5 messages should be processed in a sec over all the subscribers in multiple pods.

Similar to the implementation that is provided by Bull Queue, i.e.

It is possible to create queues that limit the number of jobs processed in a unit of time. The limiter is defined per queue, independently of the number of workers, so you can scale horizontally and still limiting the rate of processing easily:

// Limit queue to max 5 jobs per 1 second.
const myRateLimitedQueue = new Queue('rateLimited', {
  limiter: {
    max: 5,
    duration: 1000
  }
});
EldertGrootenboer commented 2 years ago

Currently we only do throttling on a namespace level, using credits in the basic and standard tiers, and MUs in the premium tier. We don't have any immediate plans to implement this on an entity level, but please create add this to Azure Feedback, and we might revisit this in the future if there is more demand for this. Please make sure to include the scenario where you would like to use this for, as it helps us give prioritization to the various items.

namansukhwani commented 2 years ago

Thanks for the reply!