tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.75k stars 527 forks source link

Question about partitionMetadata #1716

Open katlim-br opened 1 month ago

katlim-br commented 1 month ago

Is your feature request related to a problem? Please describe. I want to know if the partitionMetadata that the partitioner receives is always up to date.

Additional context I'm new to kafka and kafkajs, and I have this doubt.

For example, if I have a running producer process that emits messages to a topic against e.g. 10 brokers.

And for some reason, I reduced the numbers to 8 brokers. Will the partitionMetadata the partitioner receives have the up-to-date info of 8 brokers? Or is there a "cache" that would need to be refreshed?

I guess if the number of brokers is increased from 10 to 12, even if there is a cache, at least there is no "data loss". The problem is when reducing only.