Skeleton for new Elastic Opentelemetry distro agents

[jackshirazi]

Original description below. Per comments this is now relevant to all the new otel distros

---

Description

We're adding a new APM Java agent based on the OpenTelemetry agent, so it will be a distribution of the OpenTelemetry Java agent with additional Elastic extensions. We'll aim to generate documentation within the repo (elastic-otel-java) which should be deployed to the website similarly to the other Elastic agents - but if there's a better way, we're happy to adopt that. The existing Elastic Java agent documentation is here and there's the question about how this should be displayed in relation to that. This is a completely new agent, but it's also a Java agent, just a different implementation

Resources

https://github.com/elastic/observability-dev/issues/2882

Which documentation set does this change impact?

Integrations

Feature differences

n/a

What release is this request related to?

N/A

Collaboration model

The engineering team

Point of contact.

Main contact: @jackshirazi

Stakeholders: @elastic/apm-agent-java

[bmorelli25]

Hi Jack. My team would be happy to help with this. A few dumb questions to kick things off:

Where should the new docs live?

• Option 1: Add documentation for the new agent to the existing APM Java Agent book.
• Option 2: Create a new documentation book exclusively for the new APM Java agent.

Based on what I know about this project, I think the new Otel agent is significantly different from the existing agent, which makes option 2 the only viable option. Does that sound right to you? If we establish this pattern for the Java Otel Elastic agent, is it a pattern we can follow for the other language otel agents?

Where should the new "book" live?

Our current document system has the concept of "books". In Observability, we have 13 books—one for Observability, ten for APM agents, and two for extensions. You can see an overview of all Observability books on this page.

I'm assuming that because these are different products, we want to distinguish between pure Elastic APM agents and Elastic OTel agents. Is that correct? Ignoring the likely incorrect naming, something like this?

The other way to go about this would be to add the OTel distributions to the same list:

If this makes more sense to you, I'd be curious to understand why.

Looking forward

The book-based architecture of our current documentation system is very limiting and leads to a segmented documentation experience. In late April/early May, we're going to migrate Observability documentation to our new system. This system completely does away with book-based architecture and allows us to better connect and display different documentation sets. For example, here's a look at what the high-level Observability documentation experience will be:

On a single page, a user will be able to navigate between APM agent documentation sets and all other relevant Observability documentation, like the Observability Guide, without losing navigation context.

Since we're just a month or two away from living in this new documentation world, I'd like to keep the information architecture of the new system in mind as we make decisions about where this content will live in our current documentation system.

I have a couple of additional questions that I'll ask in a private channel.

[jackshirazi]

Based on what I know about this project, I think the new Otel agent is significantly different from the existing agent, which makes option 2 the only viable option. Does that sound right to you? If we establish this pattern for the Java Otel Elastic agent, is it a pattern we can follow for the other language otel agents?

Yes, I agree it's a significantly different agent. All of the options will be different (and we'll need to explain to users how to adapt old options to new options and the incompatibilities they have)

something like this?

I agree. Maybe the other way round, the future is the otel distros

[bmorelli25]

I was talking with Trent this afternoon and he brought up an intriguing third option: organize the nav by language. I think this option makes a lot of sense. Here's what it might look like after our migration to Docsmobile:

[stevejgordon]

I was talking with Trent this afternoon and he brought up an intriguing third option: organize the nav by language.

I like this idea as it's quicker for a language developer to find all relevant content. It also aligns more closely with how the OTel docs are organized.

[trentm]

It also aligns more closely with how the OTel docs are organized.

Good guess on where I got the idea. :)

[trentm]

@bmorelli25 If we can pull that off for the new docs, then that would be great.

For the current docs, I think either would be fine (especially given these docs may be short-lived), but I would bias to your first suggestion of separating the "Elastic APM agents" block from the "Elastic OpenTelemetry language distributions" block.

[bmorelli25]

For the current docs, I think either would be fine (especially given these docs may be short-lived), but I would bias to your first suggestion of separating the "Elastic APM agents" block from the "Elastic OpenTelemetry language distributions" block.

That's what I'm thinking, too. Since we're so close to migration, there's no point in rearranging the current docs. I've opened a PR that will add a new section above the existing APM Agent section for OpenTelemetry Distributions. Then, after we've migrated to the new documentation system, we'll break apart the content as demonstrated in https://github.com/elastic/observability-docs/issues/3558#issuecomment-2021803984.

That brings up the next topic: naming. Have we already identified a pattern for naming these new agents/distributions/libraries/tools? @trentm pointed out the following:

https://opentelemetry.io/docs/languages/ calls these variously "OpenTelemetry code instrumentation support for [language]", "A language-specific implementation of OpenTelemetry in [language]", etc. For some languages (e.g. Java, sometimes Python) the term "agent" meaningfully applies to one part of this language support.

[trentm]

That brings up the next topic: naming. Have we already identified a pattern for naming these new agents/distributions/libraries/tools?

Yes. At least us dev-ies discussed this on our weekly group call this week.

Required elements for disambiguation:
1. Elastic
2. OpenTelemetry (or OTel for brevity/less formality)
3. (Language)
4. Distribution (or “distro” for brevity or with less formality)

To disambiguate one of these things requires all four of those elements. For example the "Elastic OpenTelemetry Node.js Distribution". Current doc context might mean some could be elided. We didn't specify whether the order is strongly required. E.g. "Elastic .NET OpenTelemetry Distro" is just fine, IMHO.

The "(Language)" component is to disambiguate both (a) between the language distros and (b) distros of other OTel components, like the OTel Collector.

I am morbidly curious if:

E-las-tic O-pen-te-le-me-try Node-dot-j-s Dis-tri-bu-tion
1   2   3 4   5  6  7  8   9   10  11 . .  14  15 16   17

17 syllables is a record for a "product" name.

[bmorelli25]

That is a mouthful, but I see why all four components are included in the name. Thanks. It also looks like other folks came to the same conclusion (although they also include for):

• Honeycomb OpenTelemetry Distro for Node.js
• Splunk Distribution of OpenTelemetry for Node.js
• Lumigo OpenTelemetry Distro for Node.js

I must admit, having the language name at the end does IMO help it stick out more:

Elastic Node.js OpenTelemetry Distribution
Elastic OpenTelemetry Distribution for Node.js

[trentm]

I must admit, having the language name at the end does IMO help it stick out more:

Works for me.

[stevejgordon]

@bmorelli25 What are the next steps to move this forward?

[bmorelli25]

Are there docs that are ready to be published? If so, I can set up the infra side of things and add the otel repo to our doc build.

[stevejgordon]

@bmorelli25 We have initial getting started docs for .NET that's ready. It relates to our alpha release, so it lives in the main at the moment. If we need to cut a version branch, we can do so.

[colleenmcginnis]

All OTel repos have docs. 🎉 We also aligned on naming and terminology, and I opened a PR to add guidelines to our internal docs.