open-telemetry / opentelemetry-cpp-contrib

https://opentelemetry.io/
Apache License 2.0
125 stars 137 forks source link

Httpd traces missing service name #82

Open jvilhuber opened 2 years ago

jvilhuber commented 2 years ago

Describe your environment Describe any aspect of your environment relevant to the problem, including your platform, build system, version numbers of installed dependencies, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main branch.

Running a compiled-from-source apache httpd 2.4.51 (latest), with the downloaded mod-otel (0.1.0):

Configs:

OpenTelemetryExporter   otlp
OpenTelemetryEndpoint otel-collector.default.svc.cluster.local:4317
OpenTelemetryPropagators trace-context
OpenTelemetryIgnoreInbound off
OpenTelemetrySetResource service.name test.service

I configured the otel-collector to push traces to a grafana-tempo instance as well as a jaeger server:

receivers:
  jaeger:                            # the receives all come from the OpenTelemetry collector.  more configuration information can
    protocols:                       # be found there: https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver
      thrift_http:                   #
      grpc:                          # for a production deployment you should only enable the receivers you need!
      thrift_binary:
      thrift_compact:
  otlp:
    protocols:
      grpc:
      http:
processors:
  batch:
extensions:
  health_check:
  pprof:
  zpages:
exporters:
  otlp:
    endpoint: tempo-otlp.observability.example.com:443
  jaeger:
    endpoint: "jaeger.example.com:14250"
    tls:
      insecure: true
service:
  extensions: [health_check,pprof,zpages]
  pipelines:
    traces/1:
      receivers: [otlp,jaeger]
      processors: [batch]
      exporters: [otlp,jaeger]
  telemetry:
    logs:
      level: "info"
  1. In tempo, the traces seem to exist but show absolutely no data at all.
  2. In jaeger, The trace shows up, but jaeger says Service: OTLPResourceNoServiceName

Side-note: I can't tell what resource the otel module is sending, as the build doesn't seem to use the later opentelemetry-cpp library (1.0.0) which should include https://github.com/open-telemetry/opentelemetry-cpp/pull/906 So it's hard to say what's going on here. As far as I can tell (newbie to telemetry!) service.name resource is required, but I suspect it's not being sent.

Steps to reproduce Describe exactly how to reproduce the error. Include a code sample if applicable.

apache-http 2.4.51 + mod-otel 0.1.0 -> otel-collector (otlp) -> tempo(otlp) and or jaeger

Given the right opentelemetry-cpp ostream output (see library version 1.0.0 and the PR above) this should be testable via the instrumentation/http/tests in some way.

What is the expected behavior? What did you expect to see?

I expected to see valid traces in both grafana-tempo and jaeger.

What is the actual behavior? What did you see instead?

"empty" traces in tempo (no data at all) and traces with OTLPResourceNoServiceName as the service name in jaeger.

Additional context Add any other context about the problem here.

jvilhuber commented 2 years ago

I found a similar (same?) bug in the nginx module: https://github.com/open-telemetry/opentelemetry-cpp-contrib/issues/27 Perhaps this just needs a bump to the latest opentelemetry-cpp library?

jvilhuber commented 2 years ago

@TomRoSystems Any chance you could update the library to something recent to see if this fixes some of the issues I've had? I suspect https://github.com/open-telemetry/opentelemetry-cpp-contrib/issues/78 is also related to this.

jvilhuber commented 2 years ago

@TomRoSystems Any update on this one?

jvilhuber commented 2 years ago

FWIW I did a local test upgrading to opentelemetry-cpp-1.0.0-rc1 (the latest version that builds without Api changes) and it seems to fix the problem. So I'm pretty sure just upgrading the opentelemetry-cpp library will do the trick here. I got stuck on adding a default service.name, however, and since cpp isn't my language, I'm not sure any PR of mine would be good enough (though I'm happy to share what little I did i.e. hacked up).

TomRoSystems commented 2 years ago

Are you building with Bazel or CMake? As I think CMake was already using version you mentioned - setup-cmake.sh line 62

jvilhuber commented 2 years ago

I was using bazel, but cmake seems faster, so I can use that instead. Any chance you can upgrade to the latest opentelemetry (1.1.0 I think)? EDIT: How are the assets under https://github.com/open-telemetry/opentelemetry-cpp-contrib/releases built? bazel or cmake?