search

braedon / prometheus-kafka-consumer-group-exporter

Prometheus Kafka Consumer Group Exporter
MIT License
73 stars 39 forks source link

Add metric to track timestamps of commits #16

Closed jutley closed 5 years ago

jutley commented 6 years ago

This PR introduces a new metric kafka_consumer_group_timestamp_seconds, which is a gauge that contains the timestamp (scaled to seconds) of the latest commit for a group/topic/partition.

I'd like to add this because as is, we have no reliable way to determine when the latest commit is, which we would find useful. For active consumers, we can see when the kafka_consumer_group_commits metric increases, but for old consumers who do not have fresh commits, there is no such mechanism.

This can be useful to limit queries to show data that's been updated within a certain window, or to exclude alerts for old consumers that have been deprecated.

jutley commented 6 years ago

Mind taking a look at this @braedon? Happy to provide additional context or information if needed.

braedon commented 6 years ago

Thanks for the PR @jutley. I'm super busy at the moment (currently on a business trip), but taking at look at this is definitely on my TODO list!

braedon commented 5 years ago

Sorry for the massive delay in review this @jutley.

Made a few tweaks in https://github.com/braedon/prometheus-kafka-consumer-group-exporter/pull/27 - note the metric name change in particular. Will get it released shortly.

braedon commented 5 years ago

@jutley released in 0.5.0