Open simitt opened 2 years ago
Turns out that managing the meta
information in the event is not only a problem for apm, but also for the original beats implementation. This is now handled in the output implementation https://github.com/elastic/beats/pull/30055/files.
This should reduce the issues to handling onboarding and sourcemaps events. Mayb we can leverage the meta field and just always overwrite the meta.index
for these events before shipping them.
@lucabelluccini - APM doesn't make use of the new require_alias
feature yet due to reasons explained above; but we look into how to support it.
Libbeat changes have been reverted in https://github.com/elastic/beats/pull/30058. A new PR https://github.com/elastic/beats/pull/30055 is up.
New libbeat issue to follow along with: https://github.com/elastic/beats/issues/27874
Follow up from https://github.com/elastic/apm-server/pull/7101. With elastic/beats@84bf434, libbeat introduced a safeguard check before ingesting data to ES. It adds the
IsAlias()
function to theIndexSelector
Interface, enabling the output handler to require analias
instead of anindex
when ingesting data.The APM Server currently always returns
false
for theIsAlias()
check for two reasons.IsAlias
is not straight forward when returning theautoSelector
. A fist pass on this can be found in https://github.com/elastic/apm-server/pull/7101/commits/5111d3ecb430fc04686ac3623d94b79d02126a10. This implementation does not yet handle whether analias
ofindex
is set in themeta
information.Investigate how this check can be leveraged for general event ingestion, while still allowing to send sourcemap and onboarding data to indices.