Clarify the attribute truncation algorithm

[MrAlias]

The current attribute truncation algorithm specification is under-defined. This can, and has, led to equivalent data incompatibilities or inconsistently dropped user data.

Currently the specification defines the following algorithm to truncate string attribute values that exceed a defined limit:

if it is a string, if it exceeds that limit (counting any character in it as 1), SDKs MUST truncate that value, so that its length is at most equal to the limit

This definition is ambiguous in the following ways:

• The definition of a character is under-defined
  • It could be interpreted differently based on the context of a language (e.g. a bit, an ASCII character, a Unicode code point)
• The truncation operation is under-defined
  • Are suffix-es removed?
  • Are prefix-es removed?
  • Is non-beginning or end subsets of the strings removed?
  • Are invalid characters removed before one of the above operations?
  • Is there a character appended/inserted to indicate a truncation occurred?
  • What are the truncation boundaries (byte, character)?

Based on the different interpretations of these ambiguities, equivalent user data can incompatible based on different implementations or even be dropped due to the produced values being invalid for transport protocols.

[MrAlias]

Related to, and might be fixed by: https://github.com/open-telemetry/opentelemetry-specification/pull/2779

[MrAlias]

Suggested alternative originally posted by @jsuereth in https://github.com/open-telemetry/opentelemetry-specification/issues/2779#issuecomment-1244069138

I'd suggest we alter the specification slightly here. It's meant to leave implementation open to SDKs a bit. So here are two valid interpretations that I think a better for users:

• We truncate with a known terminal string and cut the string as necessary to include this string (not just at grapheme level). So Line 12345 becomes Line{tr}. Note: the string REMAINS valid UTF-8
• We truncate the ENTIRE string (acceptable by specification).

Either way I'd highly advice against returning invalid UTF-8. Not only does it violate the spec, but I believe it breaks protocol buffers.

[kenfinnigan]

Has there been any progress on this issue?

A related aspect is truncation of entire attributes once reaching the limit? The current behavior, as per the spec, is to drop any new attributes being added once the max limit is reached, with only a MAY for a log entry.

As with truncating values based on length, truncating entire attributes due to a limit is worthwhile information to be present as an attribute indicating this detail. Possibly not for metrics, but for Spans, knowing which attributes were truncated or removed is important information for users to visualize in tooling.