Client ID should not be reused across all partitions but should be unique per exporter instance

camunda-community-hub / zeebe-kafka-exporter

Export events from Zeebe to Kafka

Apache License 2.0

36 stars 27 forks source link

Client ID should not be reused across all partitions but should be unique per exporter instance #38

Closed npepinpe closed 3 years ago

npepinpe commented 4 years ago

Description

At the moment, users can configure a producer client ID via the configuration. However this is applied to all instances of the exporter - if a broker is leader of more than one partition, there will be more than one instance. The impact is low however, since it seems to be mostly affecting MBean and other JMX stuff - but to be safe we should anyway do as recommended by the Kafka documentation.

tjwp commented 4 years ago

+1 for fixing this. We saw these errors when first starting to use the Kafka exporter and it was confusing as to whether the exporter was working correctly.

npepinpe commented 4 years ago

Current idea is to let users specify a client ID prefix, and we just add the partition ID at the end, which should make it unique for the local broker.