elastic / apm-server

https://www.elastic.co/guide/en/apm/guide/current/index.html
Other
1.22k stars 525 forks source link

Make use of IsAlias safeguard introduced in libbeat #7116

Open simitt opened 2 years ago

simitt commented 2 years ago

Follow up from https://github.com/elastic/apm-server/pull/7101. With elastic/beats@84bf434, libbeat introduced a safeguard check before ingesting data to ES. It adds the IsAlias() function to the IndexSelector Interface, enabling the output handler to require an alias instead of an index when ingesting data.

The APM Server currently always returns false for the IsAlias() check for two reasons.

Investigate how this check can be leveraged for general event ingestion, while still allowing to send sourcemap and onboarding data to indices.

simitt commented 2 years ago

Turns out that managing the meta information in the event is not only a problem for apm, but also for the original beats implementation. This is now handled in the output implementation https://github.com/elastic/beats/pull/30055/files.

This should reduce the issues to handling onboarding and sourcemaps events. Mayb we can leverage the meta field and just always overwrite the meta.index for these events before shipping them.

simitt commented 2 years ago

@lucabelluccini - APM doesn't make use of the new require_alias feature yet due to reasons explained above; but we look into how to support it.

simitt commented 2 years ago

Libbeat changes have been reverted in https://github.com/elastic/beats/pull/30058. A new PR https://github.com/elastic/beats/pull/30055 is up.

simitt commented 2 years ago

New libbeat issue to follow along with: https://github.com/elastic/beats/issues/27874