Blizzard / node-rdkafka

Node.js bindings for librdkafka
MIT License
2.1k stars 390 forks source link

Increase in topic partition errors out with PARTCNT and severity = 5 #1013

Open venkateshan-niladri opened 1 year ago

venkateshan-niladri commented 1 year ago

If the number of partitions on a given topic is increased, ideally the consumer should register automatically for those partitions and start consuming the events. In our case, the consumer started consuming the events 3 days after the new partitions were added. Old events were not fetched too. Can you please provide insights on what is that config which needs to be done to automatically register the newly added partitions? We are using event.log to log the events. This is what we got in our logs :

{
    "severity": 5,
    "fac": "PARTCNT",
    "context": {}
}

Environment Information

Steps to Reproduce

node-rdkafka Configuration Settings

'group.id': "test",
'enable.auto.commit': false,
'fetch.message.max.bytes': 10048576,
'offset_commit_cb': (error: any) => {
    if (!error) return;
},
'statistics.interval.ms': 1000,

Additional context