[Discussion/IA] Move "automatic instrumentation" out of "Language APIs & SDKs"

[svrnm]

@open-telemetry/docs-approvers As a follow up to #3761 I'd like to suggest a significant change to the "Automatic Instrumentation" pages as well:

Move all "Automatic" pages that provide "true"[^1] zero-code instrumentation into a top level section that is separate from "Language APIs & SDKs"

Why?

As of today we heavily confuse end-users by not consistently using the term "automatic", it often means "automated approaches to extracting portable telemetry data with zero source code modification" (see OTEP-1, but it is also often used for throwing a bundle of instrumentation libraries into your SDK initialization and seeing a lot of telemetry dropping out of your application. Both are fair use cases for the word automatic, but as @martinjt pointed out in #3228 this is "is making people [...] a little confused about the "right" approach.", so getting rid of manual (as in #3761) was step 1 towards that goal, and getting rid of automatic is step 2.

The obvious solution would be renaming the "Automatic" category under "Language APIs & SDKs > Language", but spending some time thinking about it, this still does not solve all the problems:

• We still mix the use of Zero-Code instrumentation and code-based instrumentation throughout the API&SDK documentation, e.g. we do a getting started with zero-code but then the rest of the docs is code-based, or we have exporter pages that say "here is how you do it with code, but by the way if you come from the zero-code getting started, here's how you do it with environment variables". By separating the 2 topics we can untangle a lot of that.
• Having the Automatic/Zero-Code so close by in the content tree, gives people that are exploring what they can do with the APIs & SDKs the feeling that this is something they should read about, and that this is something they need to do, because "If I'm using Manual Instrumentation I'm doing it wrong". By separating the 2 topics we make it explicit that Code-based and Zero-Code is for different audiences, depending on their needs
• Also, technically the zero code instrumentations are not part of the APIs&SDKs. They are not covered by the spec, and for most languages they live in a dedicated repository -instrumentation. They are tools built on top of the SDKs, similar to the K8s operator which is built "on top" of the collector. By separating the 2 topics we make this much clearer to end-users, that this is something "different".
• Above I said that we also use "automatic" sometimes to talk about instrumentation libraries, and that this creates confusion where to draw the line: instrumentation libraries are first of all an intermediate help construct for the SDK until "every library and application observable out of the box" (see the spec. They can (and should) live independent of any zero code instrumentation, although they make heavy use of it. By sepearting the 2 topics we make it much clearer to end-users that they can leverage instrumentation libraries without using the zero code instrumentation

What this change contains

• We need to write some getting started guides for "manual" instrumentation (if I am not wrong, this applies to java, python and PHP), and have them in "Language APIs & SDKs > Language > Getting Started"
• Move getting started guides that are using "automatic" instrumentation right now into that new section, i.e. "Zero Code Instrumentations > Language > Getting Started" (yes, there will be 2 getting started per language then, 1 for "manual", 1 for "automatic")
• Review pages and move things around, whenever something references "Automatic"
• Add some cross-references at appropriate places
• Add some extra documentation that talks about "hybrid" use cases.

[^1]: Ruby has an "Automatic" page but it talks about an instrumentation library bundle, so it's not "true" zero code.

How it could look like

Note: The word "Zero-Code Instrumentation" is the first one that came to mind, we might consider something different. I think it's important that we find the right word to describe "automated approaches to extracting portable telemetry data with zero source code modification."

(The remaining "Automatic" in that image is a copy&paste error, ignore it)

[cartermp]

So I think in general I like this split, since I think there's often different audiences who care about this (or at least the same audience has separate concerns at different stages of their observability journey).

One thing I'll advocate for is that we include a stub for the K8s operator.

[svrnm]

So I think in general I like this split, since I think there's often different audiences who care about this (or at least the same audience has separate concerns at different stages of their observability journey).

💯 , I tried to avoid saying that the one is for devs, and the other one is for ops, because this is not exactly the truth, so I like what you say about "the same audience has separate concerns at different stages of their observability journey".

One thing I'll advocate for is that we include a stub for the K8s operator.

Good point, python is doing that already in a simple way here: https://opentelemetry.io/docs/languages/python/automatic/operator/

[svrnm]

@open-telemetry/dotnet-approvers , @open-telemetry/java-approvers @open-telemetry/javascript-approvers @open-telemetry/php-approvers @open-telemetry/python-approvers please take a look at this proposal, as it's going to affect especially your part of the documentation. What do you all think?