search

logstash-plugins / logstash-input-syslog

Apache License 2.0
37 stars 38 forks source link

Feat: ECS compatibility support #63

Closed kares closed 3 years ago

kares commented 4 years ago

Support for Elastic Common Schema

All event fields produces by the input are expected to be aligned with ECS.

resolves #60

(depends on https://github.com/logstash-plugins/logstash-patterns-core/pull/262 and https://github.com/logstash-plugins/logstash-filter-grok/pull/162)

kares commented 4 years ago

something to consider documenting (from https://github.com/logstash-plugins/logstash-patterns-core/pull/262#discussion_r459038278):

Quick note on host.name vs host.hostname. This may or may not change how you approach the parsing here.

host.hostname is meant to be the hostname as is reported by the system or syslog.

host.name is meant to default to the hostname, but be configurable. This helps in situations like managing a fleet of Macs mostly named "MacBook-Pro" :-)

The Elastic Security app uses host.name as the primary identifier for hosts. If host.name is not populated, events won't show up in SIEM, or there will be problems.

webmat commented 4 years ago

Thanks for carrying over that discussion here. In the initial comment, I was focusing on hostname, but I'd like to add another detail about SIEM's decision to use host.name as the main identifier.

The obvious reply to that is "what about host.id as the identifier?" -- a valid question.

But just like host names, host.id can sometimes have problems as well, especially in virtualized environments, or with some configuration management tools. There's many situations where unless the operator takes an explicit step to ensure the host ID is re-generated after provisioning, many hosts end up having the same host id.

Even if the "right" way to do it is to ensure host ID unicity, the reality is many environments don't do that.

So the decision was made to use host.name as the pragmatic identifier. Its default value of the hostname is often good enough (a simple and sensible default). And it's meant to be configurable when needed, with host.hostname containing the untampered hostname for completeness.