search

taskiq-python / taskiq-redis

Broker and result backend for taskiq
MIT License
40 stars 18 forks source link

Limit the number of messages that the worker tries to process at the same time #13

Closed booqoffsky closed 1 year ago

booqoffsky commented 1 year ago

It seems to use loop.create_task(callback(redis_message)) in the main worker cycle can lead to an undesirable state (the number of tasks in the worker can increase indefinitely).

Maybe it's better to change this mechanism? Correct me if I'm wrong

s3rius commented 1 year ago

I guess you're right. We can change broker API, so brokers won't control how much tasks they will be executing.

My suggestion here is to turn the listen method to async generator. So it would yield new tasks.