search

open-telemetry / opentelemetry-specification

Specifications for OpenTelemetry
https://opentelemetry.io
Apache License 2.0
3.71k stars 887 forks source link

Record-level attribute semantic conventions common for all telemetry signal types #2488

Open pmm-sumo opened 2 years ago

pmm-sumo commented 2 years ago

Problem

Specification has a dedicated place for record-level semantic conventions for each signal. Those are spread across several places, though many of the attribute-related conventions should be considered common. For example, http metrics vs spans or RPC metrics vs spans list a lot of common attributes.

Also, there are conventions provided in some areas for spans (e.g. database) which are not present for metrics, yet might be considered largely relevant. This is even more true for logs, where only media conventions are defined as of now

While each signal has its own specifics, I believe we should use the same semantic conventions for the shared attributes and keep everything else (signal-specific attributes, instrument names) consistent

Proposals

Some possible solutions that come to my mind:

  1. We could move the signal-specific semantic conventions into a new space, where all record-level attributes/instrument names/etc. would be defined. So e.g. a single file with HTTP semantic conventions (rather than two), with a table column describing which attributes are relevant for each signal.
  2. Like above but only for the shared attributes
  3. Whenever defining new semantic conventions, copy them for all signals, if relevant. Do this exercise also for all existing semantic conventions.
  4. Something else (suggestions welcome)
arminru commented 2 years ago

For each signal some signal-specific conventions are defined. There are span name, kind, links and parent/child structures for spans and metric name and instrument type for metrics. The attributes, however, should have the same semantics regardless of where they occur (this should also hold true for resource attributes since we (currently) don't have any overlap with the record-level attributes). I agree that having the attributes defined once for all signals would make sense. This is already partially done today but not in a structured approach. The metric conventions usually don't define their own attributes but reference the ones from the respective trace conventions but only select a subset of these to fulfill the requirements expected for metric attributes (low cardinality to allow for filtering and aggregation). This is pretty much the "copy over" suggestion 3 but only done on demand and not for all signals.

Both suggestion 1 and 2 should work but having them separate files/folders would allow for separate code owners to be defined as there are dedicated trace/metrics/logs-approver teams in the spec repo. The shared attributes would then be in the common folder where all teams can be designated as owners.

It has been a while since I worked on the semantic conventions generator tool (https://github.com/open-telemetry/build-tools/tree/main/semantic-conventions) but I'm certain that some adaptions would be necessary to make it work this way. I think having a convtype of "none" or "common" could be introduced for defining just the attributes and they could then be referenced using the ref notation that we already use for cross-area attributes within the trace conventions. Also support for metrics and logs is yet to be added but for metrics we already have a PR open in https://github.com/open-telemetry/build-tools/pull/79.

pmm-sumo commented 2 years ago

Related PR: https://github.com/open-telemetry/opentelemetry-specification/pull/1977 and issue with proposal how to address it: https://github.com/open-telemetry/opentelemetry-specification/issues/2397

pyohannes commented 2 years ago
  1. could move the signal-specific semantic conventions into a new space, where all record-level attributes/instrument names/etc. would be defined. So e.g. a single file with HTTP semantic conventions (rather than two), with a table column describing which attributes are relevant for each signal.
  2. Like above but only for the shared attributes

(2) sounds feasible to me, I think this is what @tedsuo proposed in #1977.

I think we also don't want to miss the ability to have a different document status for different signals. HTTP semantic conventions for traces might be "Feature-freeze", while those for metrics might still be "Experimental". This becomes more challenging as attributes are shared across signals.

reyang commented 2 years ago

+1 this is also well aligned with the well-established Elastic Common Schema.