Support map values and nested values for attributes

[tigrannajaryan]

After https://github.com/open-telemetry/opentelemetry-specification/pull/368 gets merged we will have support for array values.

If we add support for maps and nesting it will allow to represent arbitrary nested data structures in attribute values if needed.

This will apply to span and resource attributes.

[tsloughter]

I do want to once more iterate that from a user perspective not having maps as values is going to hurt the otel experience because different vendors will offer different forms of nested attributes. There should be on doubt that json is going to get used in values and eventually parsed by some vendors into attributes. I don't want to see that and think the issues raised so far aren't blockers but issues to be resolved in the PR that adds support for maps and updates the semantic conventions.

[Oberon00]

Right, I think it is the same topic (as in, instead of adding maps, declare source.endpoint.host and source.endpoint.port can/should be parsed into maps

So then the example posted by @tigrannajaryan with the mixture of dots in names and nested objects in https://github.com/open-telemetry/opentelemetry-specification/issues/376#issuecomment-1230397133 would be something that you do not want?

[Oberon00]

I do want to once more iterate that from a user perspective not having maps as values is going to hurt the otel experience because different vendors will offer different forms of nested attributes

If there is pressure from vendors to support this, ideally with prior art, that would be a good argument to support this IMHO. Any examples?

EDIT: And probably going through the OTEP process is warranted for this issue.

[cartermp]

If there is pressure from vendors to support this, ideally with prior art, that would be a good argument to support this IMHO. Any examples?

This is how Honeycomb works for non-OTLP data FWIW. It's been a feature since long before OpenTelemetry, for all event and tracing data sources. Another example that I know does this today is Seq.

[Oberon00]

It seems fine to me to introduce it as a feature for backwards-compatibility with vendors that support this. But using it in our semantic conventions could make people want to re-design them altogether (as @tsloughter mentioned one might want to replace dotted names with subobjects), and that decision should be synced with the semantic conventions SIG @jsuereth. If we decide to use nesting in our semantic conventions generally, IMHO we should also give that idea & implementations thereof more time to "bake" before we declare anything stable.

[Oberon00]

So I think there are more issues here that need decisions:

• Technical support for nested values in SDKs, APIs (needs to be added in a backwards-compatible way which might not be always trivial in all languages but should be doable), OTLP exporters, collector, OTLP backends, UIs, definition how to translate them in other protocols that OTel supports, like Jaeger, Zipkin
• Decision to use them or not in any new semantic conventions in the future. If so, requires support in the semantic convention generator/YAML, schema files, in any languages semantic convention artifacts (some use strong typing for keys, e.g. generated Java code https://github.com/open-telemetry/opentelemetry-java/blob/main/semconv/src/main/java/io/opentelemetry/semconv/trace/attributes/SemanticAttributes.java)
• Decision to use them or not in existing semantic conventions. If so, may require further changes to backends that interpret the semantic conventions in some way. Especially service.name seems important.

[tigrannajaryan]

One more use-case were map values would be beneficial: https://github.com/open-telemetry/opentelemetry-specification/pull/2841

[tigrannajaryan]

Another use-case here: https://github.com/open-telemetry/oteps/pull/219

[tsloughter]

2841 is an example where the complexity for both sides (client and backend) is increased by lack of map support. The semantic convention is also harder to understand when it has to be split into pieces where it could be that the whole, like exception.structured_stacktrace is optional but within the whole there may be those that are required in structured_stacktrace but still are marked as optional since they technically are optional in the attributes since there may be no structured_stacktrace.

[jmacd]

I support map-valued attributes. On a technical level, this will not present a significant problem for the OTel-Go SDK based on my understanding of the issues. We already admit slice-valued attributes and although there was a bug in this support, it was recently fixed, and we already have a data type (attribute.Set) which represents a unique attribute-value map. To add a Map-valued attribute, we would add the attribute.Set as one of the possible Value types. (Likewise, to add List-of-attributes-valued attributes, we would apply the technique used for primitive-valued slices.)

@Oberon00 list of what it takes to finish this support looks good: https://github.com/open-telemetry/opentelemetry-specification/issues/376#issuecomment-1232783131

[yurishkuro]

I also support maps as attribute values.

I don't completely agree with @Oberon00's list https://github.com/open-telemetry/opentelemetry-specification/issues/376#issuecomment-1232783131, I think it broadens the scope of this issue more than necessary and makes it difficult to make progress. Specifically, I am talking about the two points on deciding whether maps should be used in future or existing semantic conventions. Yes, those are important questions, but they do not need to block the API change itself, since there are use cases for map attributes that are not about semantic conventions, but about capturing app-specific data.

[tigrannajaryan]

Here is a draft PR to discuss: https://github.com/open-telemetry/opentelemetry-specification/pull/2888

[austinlparker]

Closed by #3858