search

danielqsj / kafka_exporter

Kafka exporter for Prometheus
Apache License 2.0
2.21k stars 612 forks source link

With a 3 node kafka cluster which all IPs to give in --kafka.server=kafka:9092 #245

Open pankajbsn opened 3 years ago

pankajbsn commented 3 years ago

Hello, This should be a very basic question, but I am not able find an answer in the README or in issues.

  1. I am running a 3 node kafka cluster on VMs (not K8s).
  2. I am planning to deploy kafka_exporter on a separate VM (called monitoring)
  3. Now, while starting kafka_exporter on the monitoring VM, which of the following option is better?
# Option1: Give all 3 of kafka nodes. Will metrics be duplicated?
kafka_exporter --kafka.server=kafka_node1:9092 --kafka.server=kafka_node2:9092 --kafka.server=kafka_node3:9092
# Option2: Give ANY one kafka node's IP. What if this node goes down.
kafka_exporter --kafka.server=kafka_node1:9092
# Option3: Find out kafka leader and give its IP. But what if leader changes because of leader re-election.
kafka_exporter --kafka.server=kafka_leader_node:9092
JohanElmis commented 3 years ago

As far as I understand, you should use option 1 (all Kafka brokers). I think it will automatically pick one of these - and get the metrics from that one if it's available - or choose another one if that one doesn't respond. I have 9 in my list - and no duplicates.