ConduitIO / conduit-connector-kafka

Conduit connector for Apache Kafka
Apache License 2.0
8 stars 5 forks source link

Feature: Emit group stats information #156

Open lyuboxa opened 1 month ago

lyuboxa commented 1 month ago

Feature description

It will be useful to have the group stats like lag, hw and per partition offset emitted in the log to keep track of progress.

lovromazgon commented 1 month ago

By default emitting it every 20-30 seconds would be enough at first, maybe make it configurable. In the future we could roll up the connector metrics into the Conduit metrics exposed through the /metrics endpoint by exposing a "metrics connector utility service" (similar to the schema service we are introducing in v0.11.0).