Currently, we prohibit the use of data_stream => true, and resolve the default data_stream => auto to false when ECS is effectively disabled. The primary driver of this is to prevent users from getting themselves into a situation implicitly that would cause them trouble.
We need to determine in what conditions a customer should be able enable data streams features when they also want to run the pipeline without ECS.
When running on Logstash 8, ECS is on-by-default, but that can be:
overridden for an individual plugin by providing ecs_compatibility => disabled,
or the default itself can be changed by setting pipeline.ecs_compatibility: disabled for an individual pipeline or for the process as a whole.
When ECS is disabled, the behaviour in this plugin is changed in a couple of ways:
when manage_templates => true, template defaults to a pre-ecs template
data_stream => auto resolves to false
When data streams is effectively enabled, manage_templatesdefaults to false.
Therefore, when a user explicitly asks for data_stream => true, and does not also set manage_templates => true, the effective value of ecs_compatibilityshould have no effect. We need to verify this assertion, and if it holds true, should begin allowing users who explicitly ask for data streams to operate with or without ECS enabled.
Currently, we prohibit the use of
data_stream => true, and resolve the defaultdata_stream => autotofalsewhen ECS is effectively disabled. The primary driver of this is to prevent users from getting themselves into a situation implicitly that would cause them trouble.We need to determine in what conditions a customer should be able enable data streams features when they also want to run the pipeline without ECS.
When running on Logstash 8, ECS is on-by-default, but that can be:
ecs_compatibility => disabled,pipeline.ecs_compatibility: disabledfor an individual pipeline or for the process as a whole.When ECS is disabled, the behaviour in this plugin is changed in a couple of ways:
manage_templates => true,templatedefaults to a pre-ecs templatedata_stream => autoresolves tofalseWhen data streams is effectively enabled,
manage_templatesdefaults tofalse.Therefore, when a user explicitly asks for
data_stream => true, and does not also setmanage_templates => true, the effective value ofecs_compatibilityshould have no effect. We need to verify this assertion, and if it holds true, should begin allowing users who explicitly ask for data streams to operate with or without ECS enabled.