redpanda-data / kminion

KMinion is a feature-rich Prometheus exporter for Apache Kafka written in Go. It is lightweight and highly configurable so that it will meet your requirements.
MIT License
614 stars 122 forks source link

Add Grafana dashboard for Cluster OPS #9

Closed weeco closed 5 years ago

weeco commented 5 years ago

Usecase

As cluster ops I'd like to use the provided metrics in a separate Grafana dashboard to get more information about a specific topic. Right now there's only one dashboard which was created to get information by specifying a consumer group name. Instead I'd like to choose a topic and in return the dashboard shows all information it has about the topic, e. g.:

Additionally I am interested in:

All the above listed features for a Dashboard can be created without adding any further metrics in Kafka Minion

weeco commented 5 years ago

Topics which haven't got any consumer group offsets (aka not consumed topics - exclude system topics here)

Probably it makes sense to implement a new metric for that, which exposes the number of unique consumer groups which have offsets for a topic, like that which indicates there are 8 consumer groups which have at least one non expired group offset (for at least one partition):

kafka_minion_topic_subscribed_consumer_groups_count{topic="customers"} 8