search

tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.75k stars 527 forks source link

KafkaJSNonRetriableError: Consumer group was not initialized, #1590

Open vikramdarade opened 1 year ago

vikramdarade commented 1 year ago

Hi Good day team,

Using kafka js to create a kafka consumer , have come across this issue wherein

Kafkajs will throw an error KafkaJSNonRetriableError: Consumer group was not initialized, consumer#run must be called first

and pod gets disconnected with kafka server silently, without consuming any new events, when kafka server have rebalanced the partitions which assigned to this pod to others.

Currently we are running 6 partitions for 3 pods, at the moment we are restarting the pods which connects them back to the kafka server and the pod starts consuming messages.

Any help would be appreciated.

OneCricketeer commented 1 year ago

I suggest adding instrumentation events and don't allow k8s to report your pod has healthy unless a group_join event was successful