search

RichardKnop / machinery

Machinery is an asynchronous task queue/job queue based on distributed message passing.
Mozilla Public License 2.0
7.56k stars 917 forks source link

Concurrency ignored for delayed tasks using redis as broker #569

Open NicolasWalter opened 4 years ago

NicolasWalter commented 4 years ago

Summary

I was reading the workers code for consuming redis tasks, and I found that there is a worker pool, used only by not-delayed-tasks workers: https://github.com/RichardKnop/machinery/blob/master/v1/brokers/redis/redis.go#L104

Here is the code for delayed ones: https://github.com/RichardKnop/machinery/blob/master/v1/brokers/redis/redis.go#L129

Question

Is this the expected behavior or it would be better to add another pool for delayed tasks? It would be really useful for me to be able to have more than one worker

Thanks!

wjh000123 commented 4 years ago

the same problem here, the consumption is too slow...

anhdowastaken commented 4 years ago

I have same opinion

RichardKnop commented 4 years ago

@NicolasWalter would you like to make a PR for this? I might not have to fix this in very near future so if it's critical it would be best to submit a PR.