search

DataDog / dd-sdk-ios

Datadog SDK for iOS - Swift and Objective-C.
Apache License 2.0
205 stars 123 forks source link

Trace sample rate does not work for child traces #2064

Open shagedorn opened 3 days ago

shagedorn commented 3 days ago

Describe the bug

In our app, we have automatic network tracking/tracing enabled, but also send a few custom traces. We apply the trace sample rate in several places in the Datadog setup (shortened for brevity):

let traceSampleRate = 5.0

RUM.enable(with: .init(
  …
  urlSessionTracking: .init(
    firstPartyHostsTracing: .trace(
      hosts: […],
      // apply sample rate to network tracking
      sampleRate: traceSampleRate
    )
  )
))

Trace.enable(with: .init(
  // apply sample rate to custom traces
  sampleRate: traceSampleRate,
  urlSessionTracking: .init(
    firstPartyHostsTracing: .trace(
      hosts: […],
      // apply sample rate to network traces
      sampleRate: traceSampleRate
    )
  ),
  …
))

We noticed in production that some of our custom traces show up much more often than others and suspected an issue with the sampling rate.

Reproduction steps

We were able to track it down by setting the sample rate to 0 (in our debug environment), and could see that a few traces always come through regardless. It turns out all custom traces with a parent trace are affected, regardless if we set the parent explicitly or Datadog detects it automatically. So, in pseudocode:

trace1.setActive()
  trace2.setActive()
  trace2.finish()
trace1.finish()

…would lead to trace2 escaping the downsampling.

trace1.setActive()
trace1.finish()

trace2.setActive()
trace2.finish()

…whereas this works as expected – given a sample rate of 0, neither of them would be sent.

SDK logs

-

Expected behavior

The trace sample rate should govern all traces to effectively limit cost.

We will need to disable custom traces entirely until this is fixed because it produces costs we cannot control, so we hope for a quick solution.

Affected SDK versions

2.14.2 - 2.17.0

Latest working SDK version

unknown

Did you confirm if the latest SDK version fixes the bug?

Yes

Integration Methods

SPM

Xcode Version

Xcode 15.4

Swift Version

Swift 5.9

MacOS Version

macOS Sonoma 14.7 (23H124)

Deployment Target

iOS 16

Device Information

Reproduces in simulators and devices

Other relevant information

No response