Define a process for dealing with OTel SemConv changes

[AlexanderWert]

The mapping for OTel data currently supports a certain (old) version of the semantic conventions.

With the OTel Semantic Conventions being merged with ECS and being stabilized we have to expect versions of SemConv to come soon that will introduce many breaking changes in the field names.

We need to define a process for dealing with different versions of Semantic Conventions so that we can support newer versions of SemConv while keeping backwards compatibility with older versions. The implication is that, with a given version of apm-data we should support a range of SemConv versions.

Related Info

• Semantic conventions define schema files that enumerate all the changes between versions
• the semantic conventions versions theoretically may even vary per signal (within a single connection / agent / SDK)

[axw]

The schema file format is defined at https://opentelemetry.io/docs/specs/otel/schemas/file_format_v1.1.0/

https://opentelemetry.io/docs/specs/otel/schemas/ says:

• OpenTelemetry instrumentation libraries include the OpenTelemetry Schema URL in all emitted telemetry. This is currently work-in-progress, here is an example of how it is done in Go SDK’s Resource detectors.

• OTLP allows inclusion of a schema URL in the emitted telemetry.

• Third-party libraries, instrumentation or applications are advised to define and publish their own telemetry schema if it is completely different from OpenTelemetry schema (or use OpenTelemetry schema) and include the schema URL in the emitted telemetry.

https://opentelemetry.io/docs/specs/otel/schemas/#use-cases describes several uses of schemas, including ingest-time transformation ("Collector-Assisted Schema Transformation"). If were to go this route, then we would still need to deal with different schema versions stored over time - so we should also consider query-time transformations.