open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.92k stars 2.28k forks source link

Kafka Receiver does not include topic or receiver as a label #35336

Open jhuizingh opened 5 hours ago

jhuizingh commented 5 hours ago

Component(s)

receiver/kafka

What happened?

Description

The metrics emitted by the Kafka Receiver to not contain enough labels to determine which Kafka Receiver and topic they are referring to if you use multiple instances of the Kafka Receiver. Many other receivers include a receiver label which shows exactly which receiver the metric comes from by name, but the Kafka Receiver does not.

Steps to Reproduce

Create an otel collector with the following relevant pieces

  ... irrelevent parts omitted ...
  receivers:
    kafka/first:
      topic: kafka-topic-first
    kafka/second:
      topic: kafka-topic-second
  service:
    pipelines:
      metrics/kafka:
        receivers:
          - kafka/first
          - kafka/second
  ... irrelevent parts omitted ...

Expected Result

Metric in the form:

otelcol_kafka_receiver_offset_lag {
  cluster="mycluster",
  http_scheme="http",
  net_host_name="10.110.0.242",
  net_host_port="8888",
  partition="24",
  server_address="10.110.0.242",
  server_port="8888",
  service_name="otelcol-contrib",
  service_version="0.103.1",
  url_scheme="http",
  receiver="kafka/first",
  topic="kafka-topic-first"
}

Actual Result

Metric in the form:

otelcol_kafka_receiver_offset_lag {
  cluster="mycluster",
  http_scheme="http",
  net_host_name="10.110.0.242",
  net_host_port="8888",
  partition="24",
  server_address="10.110.0.242",
  server_port="8888",
  service_name="otelcol-contrib",
  service_version="0.103.1",
  url_scheme="http"
}

Collector version

0.103.1

Environment information

Environment

AWS EKS Kubernetes v 1.30.3

Image: opentelemetry-collector-contrib Helm Chart: opentelemetry-collector-0.95.0

OpenTelemetry Collector configuration

receivers:
    kafka/first:
      topic: kafka-topic-first
    kafka/second:
      topic: kafka-topic-second
  service:
    pipelines:
      metrics/kafka:
        receivers:
          - kafka/first
          - kafka/second

Log output

N/A

Additional context

No response

github-actions[bot] commented 5 hours ago

Pinging code owners: