feature request: record metrics and export to prometheus

delta-io / kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake

Apache License 2.0

337 stars 71 forks source link

feature request: record metrics and export to prometheus #157

Open gdubya opened 7 months ago

gdubya commented 7 months ago

Apologies if this functionality already exists and I've missed it.

It would be great if the program could record metrics and then make them available to Prometheus (e.g. using the prometheus_exporter crate).

(I appreciate that I can add the functionality myself and send a PR; this task is a placeholder for discussion, in case anyone else works on it in the meantime ;))

leon-oosterwijk-akunacapital commented 3 days ago

I have this working on a private fork. trying to figure out how to share this.