Opentelementry dotnet and OTEL not work properly

[andreatosato]

How to configure OTEL collector and agent?

I'm study otel and opentelemetry-dotnet. I created my sample here but not work properly.

I have created .net registration like this:

services.AddOpenTelemetryTracing((builder) => builder
                        .AddSource("Sample")
                        .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("SampleApi").AddTelemetrySdk())
                        .AddSqlClientInstrumentation(s =>
                        {
                            s.SetDbStatementForStoredProcedure = true;
                            s.SetDbStatementForText = true;
                            s.RecordException = true;
                        })
                        .AddAspNetCoreInstrumentation(options =>
                        {
                            options.Filter = (req) => !req.Request.Path.ToUriComponent().Contains("swagger", StringComparison.OrdinalIgnoreCase);
                        })
                        .AddHttpClientInstrumentation()
                        .AddConsoleExporter()
                        .AddAzureMonitorTraceExporter(o =>
                        {
                            o.ConnectionString = "InstrumentationKey=61ac831c-6667-401f-ba62-962b20f604a1;IngestionEndpoint=https://westeurope-2.in.applicationinsights.azure.com/";
                        })
                        .AddOtlpExporter(otlpOptions =>
                        {
                            otlpOptions.Endpoint = new Uri("http://otel-collector:4317");
                        })

                    );
            AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);

I have set docker-compose like this:

  otel-collector:
    container_name: otel-collector
    image: otel/opentelemetry-collector
    volumes:
      - ./../config/otel-collector-config.yaml:/etc/otel-collector-config.yaml
    ports:
      - "1888:1888"   # pprof extension
      - "8888:8888"   # Prometheus metrics exposed by the collector
      - "8889:8889"   # Prometheus exporter metrics
      - "13133:13133" # health_check extension
      - "4317"        # OTLP gRPC receiver
      - "55670:55679" # zpages extension
    depends_on:
      - zipkin

  otel-agent:
    container_name: otel-agent
    image: otel/opentelemetry-collector
    command: ["--config=/etc/otel-agent-config.yaml", "${OTELCOL_ARGS}"]
    volumes:
      - ./../config/otel-agent-config.yaml:/etc/otel-agent-config.yaml
    ports:
      - "8887:8888"   # Prometheus metrics exposed by the agent
      - "14250"       # Jaeger grpc receiver
      - "14268"       # Jaeger http thrift receiver
      - "55678"       # OpenCensus receiver
      - "4317"        # OTLP gRPC receiver
      - "9411"        # Zipkin receiver
      - "1777:1777"   # pprof extension
      - "55679:55679" # zpages extension
      - "13133"       # health_check
    depends_on:
      - otel-collector

Agent config:

receivers:
  otlp:
    protocols:
      grpc:
      http:
  zipkin:

exporters:
  otlp:
    endpoint: otel-collector:4317
    insecure: true
  logging:
    loglevel: debug

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: 0.0.0.0:1777
  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp, zipkin]
      processors: [batch]
      exporters: [otlp, logging]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp, logging]

Collector config:

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  logging:
    loglevel: debug

  zipkin:
    endpoint: "http://zipkin:9411/api/v2/spans"
    format: proto

  newrelic:
    apikey: NRAK-LJ
    timeout: 30s
    traces:
      host_override: trace-api.eu.newrelic.com
      timeout: 20s
    metrics:
      host_override: metric-api.eu.newrelic.com
    logs:
      host_override: log-api.eu.newrelic.com

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, zipkin, newrelic]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, zipkin, newrelic]

if I test my app with Azure Monitor, Jeager and Zipkin, they log correctly, but if I want to use collector and then attach it to exporter, no information appears.

Can anyone help me? Thanks a lot

[Jaskaranbir]

.NET otlp-exporter only supports grpc for now. So use this receiver-config in collector:

receivers:
  otlp:
    protocols:
      # Remove http listener
      grpc:
        endpoint: 0.0.0.0:4317

This opens up grpc-listener on port 4317. So the service (using the endpoint http://otel-collector:4317) should now be able to connect to this.

[andreatosato]

I update sample but not work!

agent

receivers:
  otlp:
    protocols:
      grpc:

  opencensus:
  jaeger:
    protocols:
      grpc:
      thrift_http:
  zipkin:

exporters:
  otlp:
    endpoint: otel-collector:4317
    insecure: true
  logging:
    loglevel: debug

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: 0.0.0.0:1777
  zpages:
    endpoint: 0.0.0.0:55679

service:
  extensions: [health_check, pprof, zpages]
  pipelines:
    traces:
      receivers: [otlp, opencensus, jaeger, zipkin]
      processors: [batch]
      exporters: [otlp, logging]
    metrics:
      receivers: [otlp, opencensus]
      processors: [batch]
      exporters: [otlp, logging]

collector

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  logging:
    loglevel: debug
  prometheus:
    endpoint: "0.0.0.0:8889"
    namespace: prometheus
    const_labels:
      label1: value1

  zipkin:
    endpoint: "http://zipkin:9411/api/v2/spans"
    format: proto

  jaeger:
    endpoint: jaeger:14250
    insecure: true

  newrelic:
    apikey: NRAK
    timeout: 30s
    traces:
      host_override: trace-api.eu.newrelic.com
      timeout: 20s
    metrics:
      host_override: metric-api.eu.newrelic.com
    logs:
      host_override: log-api.eu.newrelic.com

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, zipkin, jaeger, newrelic]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, prometheus, newrelic]

[alefcarlos]

@andreatosato have you solved this ? I think im facing the same problem.

my docker-compose

version: "3.4"

services:
  jaeger: 
    image: jaegertracing/all-in-one
    ports:
      - "16686:16686"
      - "14268"
      - "14250"

  zipkin-all-in-one:
    image: openzipkin/zipkin:latest
    ports:
      - "9411:9411"

  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yaml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  # OpenTelemetry Collector
  otel-collector:
    image: otel/opentelemetry-collector:0.38.0
    command: ["--config=/etc/otel-collector-config.yaml", "${OTELCOL_ARGS}"]
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
      - ./output:/etc/output:rw # Store the logs
    ports:
      - "1888:1888"   # pprof extension
      - "8888:8888"   # Prometheus metrics exposed by the collector
      - "8889:8889"   # Prometheus exporter metrics
      - "13133:13133" # health_check extension
      - "4317"        # OTLP gRPC receiver
      - "55670:55679" # zpages extension
    depends_on:
      - jaeger
      - prometheus

And collector config:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"
    const_labels:
      label1: value1
  logging:

  zipkin:
    endpoint: "http://zipkin-all-in-one:9411/api/v2/spans"
    format: proto

  jaeger:
    endpoint: jaeger:14250
    tls:
      insecure: true

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, zipkin, jaeger]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, prometheus]

And i cant see any traces on zipkin/jaeger

[andreatosato]

@alefcarlos I use one sample for dotnetconf.

https://github.com/andreatosato/Photobook

[alefcarlos]

@andreatosato Im gonna try that one, my project is using netcore.31, but I will upgrade to net6

[andreatosato]

Metrics not work with zipkin and Jeager. @alefcarlos

[cijothomas]

Closing old issue as OTLP exporters has received many fixes and upgrades. Please create a new issue if this persists.

[CodeBlanch]

@andreatosato Hey we just got flagged because this issue potentially leaked an API key. Can you confirm you have revoked/regenerated the leaked one so I can close the security thing?

[andreatosato]

I destroy recorces