search

open-telemetry / opentelemetry-specification

Specifications for OpenTelemetry
https://opentelemetry.io
Apache License 2.0
3.76k stars 890 forks source link

Should tracers/loggers/meters be enabled by default #3983

Open lmolkova opened 7 months ago

lmolkova commented 7 months ago

Related to #3877 and #3867

Currently OTel does not control how traces/meters are enabled. It's usually controlled by users who install an instrumentation library (standalone or via distro) and sometimes explicitly enable/disable instrumentations or parts of them. Different languages and distros use different approaches. Logs rely on existing logging library configuration.

As OTel adoption grows, we see more native instrumentations, more instrumentation libs in the distros, and more details recorded by them and we need a mechanism to enable and disable instrumentation scopes (see #3867 for the context and #3877 for the solution).

One of the problems is picking reasonable defaults for OTel SDKs (while allowing users and distros to tune/override them)

Distros are able to mitigate most of the issues (e.g. provide good initial set of enabled instrumentations) with either of those approaches, but they still need to manually curate list of 'good' scopes for each library and have a default for the rest.

Full context in available in the https://github.com/open-telemetry/opentelemetry-specification/pull/3877#discussion_r1548291038

trask commented 7 months ago

maybe also related to #3205

tedsuo commented 7 months ago

@lmolkova I suggest that we look into the ability to use file config syntax to disable instrumentation.

In general, my opinion is that instrumentation should be turned on by default. Otherwise, users have a very hard time discovering what instrumentation would be available.

trask commented 7 months ago

Distros are able to mitigate most of the issues (e.g. provide good initial set of enabled instrumentations) with either of those approaches, but they still need to manually curate list of 'good' scopes for each library and have a default for the rest.

👍

In the Java agent distro we've more or less landed on: "instrumentations which produce INTERNAL spans should be off by default, except for scheduled job instrumentations (which produce top-level INTERNAL spans)"

In general, my opinion is that instrumentation should be turned on by default. Otherwise, users have a very hard time discovering what instrumentation would be available.

this could also be a bad experience if say Hibernate natively instrumented itself producing INTERNAL spans. maybe that's not something we need to solve though, as there would likely be pushback on Hibernate to make this kind of instrumentation opt-in (via their own mechanism?)

lmolkova commented 7 months ago

I think we can only continue the journey of everything is on by default since we're already on it and changing it would be breaking anyway. But we can make it better with new features like verbosity.

We still have the consistency problem - at least some of the languages (.NET, not sure if there are any others) do "everything off by default" https://github.com/open-telemetry/opentelemetry-specification/pull/3877#discussion_r1548254833

pyohannes commented 7 months ago

We still have the consistency problem - at least some of the languages (.NET, not sure if there are any others) do "everything off by default" https://github.com/open-telemetry/opentelemetry-specification/pull/3877#discussion_r1548254833

All languages I'm aware of do "everything off by default" if you're just using the plain OTel SDK (no distro or auto-instrumentation).

Obviously the situation is different when using a distro or auto-instrumentation. The .NET auto-instrumentation turns on a bunch of instrumentation libraries by default.

I'd agree there's a conceived inconsistency between languages where auto-instrumentation is the default use case (Java) and others where it isn't (.NET).

cijothomas commented 7 months ago

All languages I'm aware of do "everything off by default" if you're just using the plain OTel SDK (no distro or auto-instrumentation).

I have a different understanding! All languages are on-by-default, and .NET is the exception with off-by-default, strictly from plain OTel SDK!

By "on-by-default", what I mean is - traces/metrics from every tracer/meter is automatically collected, without any particular user-action. In .NET, the Meter/TracerProvider must be explicitly configured with the list of Tracers/Meters to listen to.

For the below pseudo code, .NET requires the TracerProvider be explicitly configured with AddSource("foo") for the spans from this tracer to work. In other languages I have checked (C++, Rust, Python), there is no additional config required.

fooTracer = get-tracer("foo");
fooTracer.StartSpan("myspan");
pyohannes commented 7 months ago

I have a different understanding! All languages are on-by-default, and .NET is the exception with off-by-default, strictly from plain OTel SDK!

@cijothomas You're right. For languages besides .NET it is enough to install and initialize instrumentation libraries, without additional SDK configuration. For .NET, additional SDK configuration is needed besides installing and initializing instrumentation libraries (for example AddSource or AddMeter on the SDK).

I was looking at it more from a user than from an SDK point of view: they have to install instrumentation libraries to "turn on" certain instrumentations. However, also from the user point of view my statement doesn't hold true when one takes into account native instrumentation (instrumentation in the instrumented library itself) which is enabled by default.

austinlparker commented 4 months ago

It seems like there's some general agreement that 'on by default' is the expectation, would a PR clarifying that help?

svrnm commented 1 month ago

@lmolkova please take a look