Kafka cluster health checking

[gianm]

kafka-manager could assist with health-checking of kafka clusters in one of two ways:

• Pull-based: Expose an http api with health-checks in it (probably a list of checks and their results). Some other system would poll the health-check periodically and alert if checks are failing.
• Push-based: Actively reach out when a problem is detected, through some kind of pluggable interface. Two useful out of the box implementations might be emails and just simple error logging.

I'm partial to the pull-based checks since I think they're probably easier to integrate with existing monitoring stuff. Either way, I think good stuff to monitor could be taken from the list here: https://kafka.apache.org/082/ops.html

Some of the most interesting stuff is:

• Broker availability (are they in ZK and can we connect over JMX?)
• Under-replicated partitions
• Unavailable partitions

[frossi85]

Hi Guys,

What is the state of this?? I want to plug some alert system and this will help me a lot.