search

samuelcolvin / arq

Fast job queuing and RPC in python with asyncio and redis.
https://arq-docs.helpmanual.io/
MIT License
1.98k stars 165 forks source link

[Question] `max_jobs` per job? #419

Closed dev-techmoe closed 6 months ago

dev-techmoe commented 7 months ago

It's there any way to set max_jobs param for each job?

For example, I have a crawler job for a website, I pushed 100 job to queue and only want 10 jobs running concurrently to prevent some API rate limit, but I have some job in same WorkerSettings that is not need to limit max_jobs, it's there any way to solve this?

dev-techmoe commented 6 months ago

I solved this problem by combine the queue_name prop of WorkerSettings and set the _queue_name param when enqueue job.

So fantastic library!