DataDog / dd-trace-rb

Datadog Tracing Ruby Client
https://docs.datadoghq.com/tracing/
Other
304 stars 373 forks source link

GRPC integration logging error even when tracing is disabled #3724

Open sco11morgan opened 3 months ago

sco11morgan commented 3 months ago

Current behaviour

GRPC integration logging error even when tracing is disabled. These errors are logged locally when running tests with tracing disabled.

E, [2024-06-18T14:45:33.966658 #817] ERROR -- ddtrace: [ddtrace] (/usr/local/bundle/ruby/3.1.0/gems/ddtrace-1.23.0/lib/datadog/tracing/distributed/propagation.rb:65:in `rescue in block in inject!') Error injecting distributed trace data. Cause: can't convert nil into Integer Location: /usr/local/bundle/ruby/3.1.0/gems/ddtrace-1.23.0/lib/datadog/tracing/distributed/trace_context.rb:124:in `format'

Expected behaviour

With tracing is disabled we'd expect no error logged.

As a workaround we are explicitly skipping the integration with

  c.tracing.instrument :grpc if Rails.env.production? || Rails.env.qa?

and that stops the errors from showing.

Environment

delner commented 2 months ago

@sco11morgan Thanks for the report! Sorry for getting back to you late.

From a quick look, to me, it looks like distributed tracing is attempting to propagate trace IDs, but since tracing is disabled, the trace ID is nil, which causes the computation to fail.

It also looks like we merged (what could be) a fix to this in our master branch for our 2.x series via #3730, possibly in response to this report. Maybe we just need to backport this to 1.x? @TonyCTHsu @marcotc WDYT?

bernardobarreto commented 1 day ago

I'm having this issue while using version 1.23. We do slow upgrades, so it would be nice if guys could backport it