search

open-telemetry / opentelemetry-collector-contrib

Contrib repository for the OpenTelemetry Collector
https://opentelemetry.io
Apache License 2.0
2.97k stars 2.31k forks source link

[exporter/elasticsearch] When using `mapping.mode: ecs`, drop scalar fields in case of object/scalar conflicts #32745

Open ycombinator opened 5 months ago

ycombinator commented 5 months ago

Component(s)

exporter/elasticsearch

Is your feature request related to a problem? Please describe.

When mapping.mode: ecs is used, certain OTel + SemConv fields are converted to ECS fields to ensure that dashboards and other assets currently using ECS fields continue to work. Due to this conversion, it's possible to run into object/scalar conflicts in the source field set, for example, {"foo": 1, "foo.bar": 2}.

Describe the solution you'd like

In such cases, we should drop the scalar value. For example, {"foo": 1, "foo.bar": 2} should be converted to {"foo": { "bar": 2}}.

Describe alternatives you've considered

No response

Additional context

No response

ycombinator commented 5 months ago

cc: @axw

github-actions[bot] commented 5 months ago

Pinging code owners for exporter/elasticsearch: @JaredTan95 @ycombinator @carsonip. See Adding Labels via Comments if you do not have permissions to add labels yourself.

crobert-1 commented 5 months ago

This was filed by a code owner, so I'm removing needs triage.

github-actions[bot] commented 3 months ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

axw commented 3 months ago

This is still relevant. I think it's only relevant with the config dedup: false, but need to confirm that. I'll take a stab at it as part of https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33776

axw commented 3 months ago

It's only relevant in the case of dedup: false, which I'm removing in https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/33776. I've added a test case to that PR.

The existing behaviour is that {"foo": 1, "foo.bar": 2} will be translated to {"foo": {"value": 1, "bar": 2}} with de-dotting. If the input is rather {"foo": 1, "foo.value": 2}, then the result will be {"foo": {"value": 2}}.

I'm not a fan of this magical ".value" addition, but inclined to leave it alone for now.

github-actions[bot] commented 1 month ago

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.