search

joowani / kq

Kafka-based Job Queue for Python
http://kq.readthedocs.io
MIT License
572 stars 24 forks source link

Question: Parallel working on Jobs #19

Open Semo opened 2 years ago

Semo commented 2 years ago

I have the following use case: Read from a arbitrary Kafka Topic and enqueue it into your queue. The requested callback parameter of Queue.enqueue method will invoke a separate class, which creates a web service call. The web service will return eventually and I'll have a result. The time needed may be between 1-10 s.

Does kq do threaded work automatically, to increase the frequence of web service calls? Or do I have to do something additional?

joowani commented 2 years ago

Hi @Semo,

Both queueing (producing) and dequeuing (consuming) is synchronous and do not use threads in the current implementation. You would have to do the threading yourself in the callback.