open-telemetry / opentelemetry-java

OpenTelemetry Java SDK
https://opentelemetry.io
Apache License 2.0
1.99k stars 827 forks source link

Metrices : Value Recorder and Observer not being exported to Otel Collector. #3188

Closed RSoodAMAT closed 3 years ago

RSoodAMAT commented 3 years ago

Describe the bug Unable to view the the observer and value recorder metrices in Prometheus.

Steps to reproduce

  1. Start the otel colltector along with jaeger and prometheus.
  2. Add the metrices for counter, recorder and observer in application
  3. Unable to view recorder and observer in prometheus.
  4. Configuration used for docker-compose for otel -collector : See Example in the end.

What did you expect to see? Should be able to view all the metrices in prometheus

What did you see instead? Saw only counter metrices in the Prometheus. Recorder and observer not showing in prometheus.

What version and what artifacts are you using? opentelemetry-api, and opentelemetry-sdk Version: used is 1.0.1 How did you reference these artifacts? Using build.gradle as shown below implementation platform("io.opentelemetry:opentelemetry-bom:1.0.1") implementation platform("io.opentelemetry:opentelemetry-bom-alpha:1.0.0-alpha") implementation("io.opentelemetry:opentelemetry-api") implementation('io.opentelemetry:opentelemetry-extension-annotations') implementation("io.opentelemetry:opentelemetry-api-metrics") implementation("io.opentelemetry:opentelemetry-sdk-metrics")

Environment OS: (e.g., "Ubuntu 18.04") Runtime (if different from JDK 11)

Example

java -javaagent:opentelemetry-javaagent-all.jar -Dotel.exporter.otlp.endpoint="http://localhost:4317" -jar -Dotel.resource.attributes=service.name=tc-job TCapp.jar

static final Meter meter = GlobalMetricsProvider.getMeter("Train Center", "1"); // Observer example class Status{ private double avgDownloadSpeed; // corresponding getter and setter }

private Status status; meter .doubleValueObserverBuilder("avg.download.speed.per.wafer") .setDescription("Average Download speed per wafer") .setUnit("mb/sec") .setUpdater( doubleResult -> doubleResult.observe( status.getAvgDownloadSpeed(), Labels.of("wafer", mapID, "tool", toolId, "recipe", recipe, "unit", "mb/sec")));

// Recorder example long timeTaken = System.currentTimeMillis(); meter .longValueRecorderBuilder("total.time.taken.indexing.task.per.wafer") .setDescription("Total time taken by dl hub indexing task per wafer") .setUnit("ms") .build() .record( timeTaken, Labels.of("wafer", waferId, "tool", toolId, "recipe", recipe, "unit", "ms"));

//Docker compose file - Bringing up otel collector , prometheus , jaeger using the otlp exmaple given in git . version: "2" services:

Jaeger jaeger-all-in-one: image: jaegertracing/all-in-one:latest ports:

Zipkin zipkin-all-in-one: image: openzipkin/zipkin:latest ports:

Collector otel-collector: image: ${OTELCOL_IMG} command: ["--config=/etc/otel-collector-config-demo.yaml", "${OTELCOL_ARGS}"] volumes:

Prometheus prometheus: container_name: prometheus image: prom/prometheus:latest volumes:

////////////////////////////////////// otel-collector donfig-demo. yaml

receivers: otlp: protocols: grpc:

exporters: prometheus: endpoint: "0.0.0.0:8889" namespace: traincenter const_labels: label1: value1 logging:

zipkin: endpoint: "http://zipkin-all-in-one:9411/api/v2/spans" format: proto

jaeger: endpoint: jaeger-all-in-one:14250 insecure: true

processors: batch:

extensions: health_check: pprof: endpoint: :1888 zpages: endpoint: :55679

service: extensions: [pprof, zpages, health_check] pipelines: traces: receivers: [otlp] processors: [batch] exporters: [logging, zipkin, jaeger] metrics: receivers: [otlp] processors: [batch] exporters: [logging,prometheus]

///////////////////////////////////////// prometheus.yaml

scrape_configs:

job_name: 'otel-collector' scrape_interval: 2s static_configs: targets: ['otel-collector:8889'] targets: ['otel-collector:8888']

jkwatson commented 3 years ago

Since we determined that this is an issue in the collector, please open this issue in the collector repo. The issue is not that they aren't getting exported to the collector. The issue is that the collector's prometheus implementation isn't processing them properly.