search

yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters
Apache License 2.0
11.84k stars 2.5k forks source link

Kafka Consumer Lag metrics for SSL kafka #528

Open 1kaushik1 opened 6 years ago

1kaushik1 commented 6 years ago

I am a bit new to kafka. I am using kafka_2.11-0.11.0.1. For security reasons we have enabled SSL features i.e SSL listeners on port 9093. I have a 3-node kafka and 3 zookeeper setup. I am trying to consume via a different linux machine by specifying the --bootstrap-server info as that of the Kafka servers. I am able to produce and consume successfully. So when we talk about monitoring, i am using JMX values which is working good for me at the kafka server level. So i am getting the necessary kafka.controller and kafka.server metrics. The only issue i am facing is with Kafka.consumer metrics. I tried one of the consumer lag exporters but with no luck. I have installed the exporter on the Kafka server,but its not giving me any output. I read a few issues reported related to the same, so is it true that the consumer information cannot be made available with an SSL kafka listening on port 9093? Also is the consumer metrics stored at the consumer side which in my case would be the linux servers through which i am consuming?

Thankyou for your help.

montana-ua commented 6 years ago

I've built 1.3.3.19 and updated one of my clusters. So, now it works! Great stuff!