search

jessetane / queue

Asynchronous function queue with adjustable concurrency
MIT License
762 stars 66 forks source link

Feature request: request throttling supporting Redis #104

Closed thecatontheflat closed 1 year ago

thecatontheflat commented 1 year ago

Hi, thanks for the great library!

I was wondering if adding Redis support for cluster-wide throttling (when using in a kubernetes deployment) could be a valid addition to this library?

jessetane commented 1 year ago

Can you give some more background on what you mean? I know that Redis is a database and that Kubernetes is a container orchestration tool, but why should this intentionally abstract and general purpose module need to know about these things?

thecatontheflat commented 1 year ago

Redis was just an example, I could have phrased it better.

The idea was to add a bulit-in feature that would enable this module to work across multiple servers. One example is when making requests to a private API that imposes certain rate limits based on the API key. In such scenario, when using queue in a multi server environment (e.g. load balanced application server), each server is only throttling its own requests without regarding the whole cluster (and therefore, the globally imposed rate limit). If this module could track rate limits "globally" (would need some sort of state persistence, e.g. Redis), it would have solved the cluster wide throttling problem.

I understand that this is probably beyond the scope of intentionally abstract and general purpose.