Blizzard / node-rdkafka

Node.js bindings for librdkafka
MIT License
2.11k stars 392 forks source link

Local: Erroneous State #1022

Closed ursulacparker closed 1 year ago

ursulacparker commented 1 year ago

Environment Information

Steps to Reproduce Manually assign a consumer to a topic and partition, then run a producer at the same time. It will randomly appear (not often). If I restart the consumer it goes away.

node-rdkafka Configuration Settings 'rebalance_cb': function(err : LibrdKafkaError, assignment : Assignment) { if (err.code === CODES.ERRORS.ERR__ASSIGN_PARTITIONS) { } else if (err.code == CODES.ERRORS.ERR__REVOKE_PARTITIONS){ } else { // We had a real error console.error(err); } }

Additional context C:\Users\user\project\node_modules\node-rdkafka\lib\kafka-consumer.js:263 this._client.assign(TopicPartition.map(assignments)); ^

Error: Local: Erroneous state at KafkaConsumer.assign (C:\Users\user\project\node_modules\node-rdkafka\lib\kafka-consumer.js:263:16)

constantind commented 1 year ago

This error is to prevent duplicate streaming from the same partition as default offset is at the broker so which one of two would commit? This happens since 1.9.0 see https://github.com/confluentinc/librdkafka/releases/tag/v1.9.0 when you try to call assign more than once to the same topic/partition. if you have more than one partition default rebalance https://github.com/Blizzard/node-rdkafka/blob/master/lib/kafka-consumer.js#L66 should call unassign automatically, however if you have single partition topic that does not trigger and you have to call unassign yourself otherwise second call to find.assignments or assign to the same partition throws that