search

danielqsj / kafka_exporter

Kafka exporter for Prometheus
Apache License 2.0
2.1k stars 602 forks source link

Cannot get consumer group: dial tcp x.x.x.:9094: connect #346

Open CodeWithAdarsha opened 1 year ago

CodeWithAdarsha commented 1 year ago

I am also facing same issue , Not able to see any consumer group information in metric API , I can See all metric related topic , broker , partitions only issue while connection consumer having issue, Below is my logs

you can see complete docker file here : https://github.com/CodeWithAdarsha/kafka-demo/blob/main/sample-kafka

2022-10-27T06:53:38.534282962Z E1027 06:53:38.533990 1 kafka_exporter.go:438] Cannot get oldest offset of topic __consumer_offsets partition 12: dial tcp 127.0.0.1:9092: connect: connection refused 2022-10-27T06:53:38.539813964Z E1027 06:53:38.539226 1 kafka_exporter.go:426] Cannot get current offset of topic __consumer_offsets partition 13: dial tcp 127.0.0.1:9094: connect: connection refused 2022-10-27T06:53:38.544272406Z E1027 06:53:38.543964 1 kafka_exporter.go:438] Cannot get oldest offset of topic __consumer_offsets partition 13: dial tcp 127.0.0.1:9094: connect: connection refused 2022-10-27T06:53:38.549107986Z E1027 06:53:38.548931 1 kafka_exporter.go:426] Cannot get current offset of topic __consumer_offsets partition 14: dial tcp 127.0.0.1:9092: connect: connection refused 2022-10-27T06:53:38.555153677Z E1027 06:53:38.554910 1 kafka_exporter.go:438] Cannot get oldest offset of topic __consumer_offsets partition 14: dial tcp 127.0.0.1:9092: connect: connection refused 2022-10-27T06:53:38.557113683Z E1027 06:53:38.556874 1 kafka_exporter.go:552] Cannot get consumer group: dial tcp 127.0.0.1:9094: connect: connection refused 2022-10-27T06:53:38.558420789Z E1027 06:53:38.558012 1 kafka_exporter.go:552] Cannot get consumer group: dial tcp 127.0.0.1:9092: connect: connection refused

jsachnet commented 1 year ago

@CodeWithAdarsha I was having the same error with the consumer group and didn't understand why kafka-exporter was trying to connect to 127.0.0.1 instead of the IP that the hostname resolved to in the command flag --kafka.server=kafka-0:9092

The only place in my docker-compose file that 127.0.0.1 was configured was in the KAFKA_CFG_ADVERTISED_LISTENERS for each kafka broker. My guess was that kafka is sending that configured IP/hostname back to a connected client which the client then tries to use so I changed my kafka environment var for the advertised listeners to include the hostname instead of 127.0.0.1 and it worked.

KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka-0:9092