search

nodefluent / node-sinek

:tophat: Most advanced high level Node.js Kafka client
MIT License
290 stars 52 forks source link

Gracefully shutdown #136

Open ht-jo opened 5 years ago

ht-jo commented 5 years ago

Hello.

I'm wondering of how to terminate a consumer gracefully, for instance, syncEvent is still running, but consumer.close() has called.

I couldn't find what waits for running tasks until finishing below code.

https://github.com/nodefluent/node-sinek/blob/462738c7e504995e8ab006513da0af174172b4c9/lib/librdkafka/NConsumer.js#L962-L981

Is there any code of termination gracefully in rdkafka or something else? Or does it just let them terminate without graceful termination? or would I implement code that clean the resources before shutdown for myself?

Thank you!

Pruxis commented 4 years ago

@krystianity is this something you want to support in the driver itself? As currently we are holding promises in memory and use consumer.pause() to stop listening to message to then await all ongoing promises before gracefully shutting down our process.

krystianity commented 4 years ago

Sorry for the late response. I am not 100% certain, https://github.com/Blizzard/node-rdkafka/issues/5 but it looks like node-rdkafka's dispatcher threads should take care of this automatically, as long as your application receives callbacks.

krystianity commented 4 years ago

Since sinek uses the HighLevelProducer you should only make sure that your produce acknowledgement settings allow for acks.