DataDog / dd-opentracing-cpp

Datadog Opentracing C++ Client
Apache License 2.0
40 stars 40 forks source link

[v1.1.4] Not able to drop a trace after root span context is accessed with default RuleSampler #136

Open tomthomson opened 4 years ago

tomthomson commented 4 years ago

With the new rules sampler in v1.1.4 I'm having problems to manually drop/keep a trace after a child span was created (and the root context has been accessed). I'm using the default tracer with a default RuleSampler+PrioritySampler.

I created the following test to demonstrate my problem. Currently the test fails because I cannot set the sampling priority after the root span context was accessed.

TEST_CASE(
    "sampling behaviour without mock sampler: sampling priority can be set until the root trace "
    "finishes") {
  datadog::opentracing::TracerOptions tracer_options{"localhost", 8126, "engine", "custom",
                                                     "develop"};
  tracer_options.sampling_rules = R"([])";
  tracer_options.sample_rate = std::nan("");
  tracer_options.priority_sampling = false;
  ot::Tracer::InitGlobal(datadog::opentracing::makeTracer(tracer_options));

  ot::StartSpanOptions start_options;
  auto root_span = ot::Tracer::Global()->StartSpanWithOptions("request", start_options);
  auto child_span =
      ot::Tracer::Global()->StartSpan("operation", {opentracing::ChildOf(&root_span->context())});
  child_span->Finish();

  root_span->SetTag("sampling.priority", static_cast<int>(SamplingPriority::UserDrop));
  auto p = static_cast<Span*>(root_span.get())->getSamplingPriority();
  REQUIRE(p);
  REQUIRE(*p == SamplingPriority::UserDrop);

  root_span->Finish();
}

The test section "sampling priority can be set until the root trace finishes" in propagation_test shows that this is possible if there is no default sampling priority set.

I wonder how I can achieve the same with the default RulesSampler.

After debugging a bit it I see the following: When root span context is retrieved to create a child span, the sampling priority is locked in WritingSpanBuffer::setSamplingPriorityImpl, because sampling with the default PrioritySampler results in a default_samplerate of 1.0 (i.e. SamplerKeep)

cgilmour commented 4 years ago

Hi @tomthomson thanks for reporting this!

It seems that test with the mock sampler is wrong, and I'll need to fix it.

In general, it should be possible to update the sampling priority of a root trace until either

I'll see what can be done to change it.