search

samuelcolvin / arq

Fast job queuing and RPC in python with asyncio and redis.
https://arq-docs.helpmanual.io/
MIT License
1.98k stars 165 forks source link

Time based Queue. Handling X jobs per minute. #453

Open waseemhnyc opened 3 weeks ago

waseemhnyc commented 3 weeks ago

How could you limit the number of jobs you are processing per a set time frame. For example I'd only like to have 60 jobs run per minute. This is helpful if you'd like to do something where you track the number of API calls you are making to avoid running in to rate limiting issues.

JonasKs commented 3 weeks ago

There's a few ways to solve this, either by limiting a worker max concurrent jobs, or by using libraries such as redis-rate-limiters.