Open jsoriano opened 4 months ago
Is this issue https://github.com/elastic/elastic-package/issues/1066 similar to the first task mentioned in the description ? @jsoriano
[elastic-package] If a version is set in _dev/build/build.yml, elastic-package system tests must check that ecs.version is set to this version or greater. Package developers may need to add pipeline processors to satisfy this check.
Yes! Good finding, I will link this issue here. Thanks.
Thank you for creating the issue. This would be a pre-requisite for having a complete solution for adopting ecs@mappings for Integrations.
- [elastic-package] If a version is set in
_dev/build/build.yml
,elastic-package
system tests must check thatecs.version
is set to this version or greater. Package developers may need to add pipeline processors to satisfy this check.
Would this mean that we will not be getting rid of build.yml ? In case we are using ecs@mappings, we will not need this.
But if we have to overwrite some of the ECS fields (say in case of TSDB), then we might need build.yml?
Would this mean that we will not be getting rid of build.yml ? In case we are using ecs@mappings, we will not need this.
By now build.yml
is going to stay in case we need a escape hatch to explicitly import some field. It will be also needed for packages supporting versions of the stack older than 8.13.
But if we have to overwrite some of the ECS fields (say in case of TSDB), then we might need build.yml?
Correct, build.yml
is still helpful on this case.
About the test in elastic-package - nice addition, not sure about the priority as it won't take affect on recent versions of the stack anyway (are we automatically running tests on older stack versions?)
About the second part - makes a lot of sense to me, we need to make sure the fleet-provided pipeline processor runs after the package-provided pipeline so we can "upgrade" an outdated ecs.version
property.
About the second part - makes a lot of sense to me, we need to make sure the fleet-provided pipeline processor runs after the package-provided pipeline so we can "upgrade" an outdated
ecs.version
property.
If we can set it in the Fleet final pipeline, this can be achieved.
- The value used must be the max version between the version of the stack and the version found in the document, if any.
@jsoriano : Say, stack version is 8.14, then the max would be taken as that. But our latest ecs.version is 8.11. So I think we will not be able to directly take the max of stack or document. Since ecs mappings version and stack version is not the same.
our latest ecs.version is 8.11. So I think we will not be able to directly take the max of stack or document. Since ecs mappings version and stack version is not the same.
Interesting, I thought ECS version was more coupled to the stack version. Fleet or some other component of the stack will need to know then what is the version being supported by ecs@mappings
.
@felixbarny do you know how the fleet plugin in Kibana could detect which ecs version ecs@mappings
corresponds to?
The ECS version of ecs@mappings
is always the same as the stack version.
@felixbarny As @ishleenk17 noted above, the last technical release of https://github.com/elastic/ecs is 8.11, but I guess what you are saying is that if there isn't an explicit ecs release that just means that the new ecs version is identical to the previous one?
Two examples:
ecs@mappings
is 8.14 which is completely identical to 8.11
We have different sources of ECS fields, and different places where
ecs.version
can be set. This proposal attempts to define some explicit rules about the expectations of theecs.version
field.The different sources of ECS fields definitions are, at least:
export
, where the version used is determined in_dev/build/build.yml
.import_mappings
, with no explicit versioning, but can be combined withexport
and also uses the definition in_dev/build/build.yml
.ecs@mappings
component template, since 8.13.The proposal would be focused on making the final documents to have the greater known ECS version affecting the package, with two main tasks:
_dev/build/build.yml
,elastic-package
system tests must check thatecs.version
is set to this version or greater. Package developers may need to add pipeline processors to satisfy this check.ecs@mappings
component template, it must also add a processor that setsecs.version
. The value used must be the max version between the latest version of ECS supported by the the stack and the version found in the document, if any. Notice that the versions of ECS and the stack are not fully aligned.cc @ishleenk17 @zmoog @elastic/ecosystem @kpollich