search

open-telemetry / opentelemetry-python

OpenTelemetry Python API and SDK
https://opentelemetry.io
Apache License 2.0
1.67k stars 571 forks source link

Setting OTEL_TRACES_SAMPLER_ARG=always_off results in no spans being exported #3851

Open masterkidan opened 2 months ago

masterkidan commented 2 months ago

Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main. According to https://opentelemetry-python.readthedocs.io/en/latest/sdk/trace.sampling.html setting OTEL_TRACES_SAMPLER_ARG=always_off should turn off sampling allowing all the spans to go through to the collector. However what actually ends up happening is that no spans are sent. This needs a clarification in the docs or some update in the behavior of that setting. Per https://github.com/open-telemetry/opentelemetry-python/issues/3474#issuecomment-1771355829, it seems like the expected behavior this effectively stops spans from being exported.

Steps to reproduce Set OTEL_TRACES_SAMPLER_ARG=always_off

What is the expected behavior? What did you expect to see? Expect all samples to be sent to collector

What is the actual behavior? What did you see instead? No spans are sent to the collector

Additional context Version: 0.44b0

xrmx commented 2 months ago

@masterkidan that is matching the specification wording so it looks like everything is fine, see https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers

masterkidan commented 2 months ago

Thanks! Can you possibly update the documentation here: https://opentelemetry-python.readthedocs.io/en/latest/sdk/trace.sampling.html Specifically this wording:

The tracer sampler can also be configured via environment variables OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG (only if applicable). The list of built-in values for OTEL_TRACES_SAMPLER are:

always_on - Sampler that always samples spans, regardless of the parent span’s sampling decision.

always_off - Sampler that never samples spans, regardless of the parent span’s sampling decision.

That's what confused me earlier. It seems to imply behavior that is different to what is mentioned in the spec.

xrmx commented 2 months ago

@masterkidan Feel free to open a PR

emdneto commented 2 months ago

I think the documentation is fine. But, what could be improved is:

always_off - Sampler that never samples spans, regardless of the parent span’s sampling decision. No spans will be recorded (i.e., DROP decision)

This should reduce confusion for users who may be unfamiliar with the terminology