Open fgierlinger opened 2 years ago
ebeahan
commented
2 years ago Even though the Logstash docs refer to the cef.* fields as part of ECS, I think they're intended to be custom fields populated alongside ECS. It's recommended to place custom fields into a namespace using a proper name when mapping to ECS.
@kares @yaauie @karenzone Is my viewpoint correct here? Should the CEF plugin doc somehow distinguish between ECS-defined and custom fields?
yaauie
commented
2 years ago While ECS does specify a specific set of interoperable fields, it also "provides a set of naming guidelines for adding custom fields", and it is in this sense that the cef-prefixed fields are fields that comply with the guidance of ECS for custom fields and so I consider them "ECS Fields".
I'm not sure if it would add noise or clarity to change the heading column of those two tables from "ECS Field" to "ECS Compatible Field"?
Or perhaps to add a qualifying statement:
When decoding in an ECS Compatibility mode, the ECS Fields are populated from the corresponding CEF Field Names or CEF Keys found in the payload’s extensions. +Fields that do not have a formal ECS definition are expanded into the `cef` namespace +to avoid clashing with non-CEF fields.
ebeahan
commented
2 years ago While ECS does specify a specific set of interoperable fields, it also "provides a set of naming guidelines for adding custom fields", and it is in this sense that the cef-prefixed fields are fields that comply with the guidance of ECS for custom fields and so I consider them "ECS Fields".
Yes, ECS defines both a set of standard fields and guidance on how events should be structured.
I understand how users see cef.* ECS fields documented and may expect a cef.* field set to exist in ECS. So I'd vote to add a statement that the cef.* fields use a custom namespace.
leehinman
commented
2 years ago Just for context. The beats cef module has the cef.extensions as custom namespace https://github.com/elastic/beats/blob/8.2/x-pack/filebeat/module/cef/log/_meta/fields.yml#L223 and the beats decode_cef processor maps numerous cef fields to their ECS equivalents https://github.com/elastic/beats/blob/d045c17ff9146add9bb26072c2c1b535b8886d69/x-pack/filebeat/processors/decode_cef/keys.ecs.go#L20
Summary The Logstash documentation of the CEF codec is referencing to CEF ECS fields (
cef.nameandcef.versionto name a few). Since the documentation refers to them as "ECS Field" they should also be defined in the ECS.Motivation: Provide all fields referenced as "ECS Field" through the ECS repository.
Detailed Design: All field names currently referenced in the Logstash CEF Codec documentation are: