[exporter/datadog] resource name renamed when http.method and http.route is defined?

karmingc commented 1 year ago

Component(s)

exporter/datadog

What happened?

Description

It seems like the resource name is replaced when http.method is defined. http.route is appended if it exist instead of using the span name.

edit: currently using this workaround: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/1909#issuecomment-1631846359

Note: I am still looking where this is happening, so I am unsure if this is really an exporter issue. However, I added zipkin as an exporter and it doesn't seem to be the case there. I opened this ticket to get some pointers from somebody more knowledgeable of the codebase...

Steps to Reproduce

Within your instrumented application, add the http.method and http.route to the appropriate span.
Send it to the collector, which exports to datadog.

Expected Result

This should probably configurable and not default to those tags.

Actual Result

This currently defaults to the tags if they exist.

Collector version

0.81.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      grpc:
      http:

exporters:
  datadog:
    api:
      site: datadoghq.com
      key: ${env:DD_API_KEY}
  zipkin:
    endpoint: http://host.docker.internal:9411/api/v2/spans
  logging:
    verbosity: detailed

processors:
  batch:

extensions:
  health_check:
  pprof:
  zpages:

service:
  extensions: [pprof, zpages, health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [logging, datadog, zipkin]
  telemetry:
    logs:
      level: "debug"

Log output

otel-otel-collector-1  | 2023-08-18T21:35:47.715Z   info    ResourceSpans #0
otel-otel-collector-1  | Resource SchemaURL:
otel-otel-collector-1  | Resource attributes:
otel-otel-collector-1  |      -> service.name: Str(svc)
otel-otel-collector-1  |      -> telemetry.sdk.language: Str(java)
otel-otel-collector-1  |      -> telemetry.sdk.name: Str(opentelemetry)
otel-otel-collector-1  |      -> telemetry.sdk.version: Str(1.25.0)
otel-otel-collector-1  | ScopeSpans #0
otel-otel-collector-1  | ScopeSpans SchemaURL:
otel-otel-collector-1  | InstrumentationScope org.springframework.boot 3.1.2
otel-otel-collector-1  | Span #0
otel-otel-collector-1  |     Trace ID       : 130cff906d92798a243731a18b4d2bff
otel-otel-collector-1  |     Parent ID      :
otel-otel-collector-1  |     ID             : 042691667c7821fb
otel-otel-collector-1  |     Name           : some name
otel-otel-collector-1  |     Kind           : Internal
otel-otel-collector-1  |     Start time     : 2023-08-18 21:35:43.60556 +0000 UTC
otel-otel-collector-1  |     End time       : 2023-08-18 21:35:43.613389666 +0000 UTC
otel-otel-collector-1  |     Status code    : Unset
otel-otel-collector-1  |     Status message :
otel-otel-collector-1  | Attributes:
otel-otel-collector-1  | Span #1
otel-otel-collector-1  |     Trace ID       : 5a0bffb2768a483243c99f465b762c56
otel-otel-collector-1  |     Parent ID      :
otel-otel-collector-1  |     ID             : a291cf906ba02ae7
otel-otel-collector-1  |     Name           : http post /api/users/jwt_token
otel-otel-collector-1  |     Kind           : Client
otel-otel-collector-1  |     Start time     : 2023-08-18 21:35:43.299011 +0000 UTC
otel-otel-collector-1  |     End time       : 2023-08-18 21:35:43.860726247 +0000 UTC
otel-otel-collector-1  |     Status code    : Unset
otel-otel-collector-1  |     Status message :
otel-otel-collector-1  | Attributes:
otel-otel-collector-1  |      -> exception: Str(none)
otel-otel-collector-1  |      -> http.path_group: Str(/api/users/jwt_token)
otel-otel-collector-1  |      -> http.method: Str(POST)
otel-otel-collector-1  |      -> http.status_code: Str(200)
otel-otel-collector-1  |    {"kind": "exporter", "data_type": "traces", "name": "logging"}

With logging, I can't see the exported traces/spans. But with an error, I can see that resource is the provided http.method:

otel-otel-collector-1  | 2023-08-18T21:46:28.159Z   debug   datadogexporter@v0.81.0/zaplogger.go:28Fixing malformed trace. HTTP status code is invalid (reason:invalid_http_status_code), dropping invalid http.status_code=CLIENT_ERROR: service:"svc"  name:"org.springframework.boot.client"  resource:"POST"
...

Additional context

No response

github-actions[bot] commented 1 year ago

Pinging code owners:

exporter/datadog: @mx-psi @gbbr @dineshg13 @liustanley @songy23 @mackjmr

See Adding Labels via Comments if you do not have permissions to add labels yourself.

github-actions[bot] commented 12 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/datadog: @mx-psi @gbbr @dineshg13 @liustanley @songy23 @mackjmr
needs: Github issue template generation code needs this to generate the corresponding labels.

See Adding Labels via Comments if you do not have permissions to add labels yourself.

mackjmr commented 12 months ago

@karmingc this is currently the expected behaviour for resource name.

I am still looking where this is happening, so I am unsure if this is really an exporter issue.

The logic that extracts the resource name can be found here (datadogexporter imports github.com/DataDog/datadog-agent/pkg/trace).

You can override this logic shared above by setting attribute resource.name.

karmingc commented 12 months ago

Thanks for the insight @mackjmr.

open-telemetry / opentelemetry-collector-contrib