search

squeaky-pl / japronto

Screaming-fast Python 3.5+ HTTP toolkit integrated with pipelining HTTP server based on uvloop and picohttpparser.
MIT License
8.61k stars 581 forks source link

python Kafka producer does not work with multi worker python framework like sanic and japronto #119

Closed souravroy0708 closed 6 years ago

souravroy0708 commented 6 years ago

I am implementing python sanic web framework to produce message in kafka topic.So when I am using worker more than one in my application then kafka produce not able to produce message but when i am using worker as 1 then kafka client able to produce the message.

So is there any way to work kafka producer with python multiprocessing.After running python app.py I am able to that it is running in multiple processes.

501 14712 14709 0 7:56AM ttys002 0:00.03 python app.py 501 14713 14709 0 7:56AM ttys002 0:00.03 python app.py

Sanic reference - http://sanic.readthedocs.io/en/latest/sanic/deploying.html#workers

squeaky-pl commented 6 years ago

This is a duplicate of https://github.com/squeaky-pl/japronto/issues/78

There is no clean way to do it / official API, you would need to attach to the loop after fork calls happen and the loop instance is initialized.