search

elastic / apm-queue

Abstraction layer over Kafka / GCP PubSub Lite to produce and consume records
Apache License 2.0
6 stars 15 forks source link

Leverage connects metrics for success and failure #479

Closed endorama closed 3 months ago

endorama commented 3 months ago

This PR address https://github.com/twmb/franz-go/issues/670 but in the monitoring implementation introduced in this module by #460.

It changes Kafka connection metrics, instead of relying on 2 different metrics to report Kafka connections successes and failures, use only a single metric with distinct outcome attribute based on the outcome. This implementation does not guard this new behaviour under a flag, like I did for the contribution to franz-go as I don't think it's useful is a specialized implementation like we have here. But still this introduces a breaking change in the kafka package monitoring so I'm open to revisit this decision if we want to keep backwards compatibility.

Metric messaging.kafka.connect_errors.count has been removed.

Attribute outcome has been added to messaging.kafka.connects.count with 2 possible values: success and failure.

endorama commented 3 months ago

Upon internal feedback this PR now does not remove a metric but only adds dimensions & emit connects metric also on failure. Removing the old metric will happen in a future update.

endorama commented 3 months ago

Added a follow up issue to keep track of removing messaging.kafka.connect_errors.count.