search

elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
112 stars 4.93k forks source link

Module dashboards should not use non-ECS `agent.hostname` fields #12983

Closed MikePaquette closed 4 years ago

MikePaquette commented 5 years ago

Background: As we transitioned beats to ECS in 6.x and 7.0, we made breaking changes to change beat.hostname to agent.hostname. However, agent.hostname is not an ECS field, and is not expected to ever become an ECS field (see https://github.com/elastic/ecs/issues/178#issuecomment-512586064)

ECS has two relevant fields host.name and host.hostname. host.name is populated for most beats, except for heartbeat (see https://github.com/elastic/beats/issues/12107) and APM, even if add_host_metadata is not enabled.

host.hostname is added when add_host_metadata is enabled.

Generally, host.name and host.hostname will contain the same information.

Describe the enhancement: Modify any dashboard, saved search, alert, ML job, or other module content or solution widget that uses agent.hostname to instead use ECS host.name.

** ~Strikethrough~ if not applicable

Note 1: If the module or solution expressly and by default enables add_host_metadata, and there is a specific need the capture the (uncommon) distinction of host.hostname vs. host.name, then host.hostname may be used. Otherwise, all use of agent.hostname should be changed to host.name.

Note 2: It is probably best to continue to populate agent.hostname in the events themselves since users have built content using it, and we don't want to make yet another breaking change for them.

Describe a specific use case for the enhancement or feature: Users are confused as to which field they should be using in their dashboards, as there has been a number of changes starting with 6.3. They hoped that they'd need to make one set of changes to get to ECS. They see agent.hostname being used in our dashboards and think it is the default, and that they should use it, but it's not an ECS field.

Related Issues: https://github.com/elastic/beats/issues/11860 https://github.com/elastic/beats/issues/11790

elasticmachine commented 5 years ago

Pinging @elastic/secops

MikePaquette commented 5 years ago

cc: @webmat ^^

webmat commented 5 years ago

Thanks Mike!

Note 2 addendum: we should leave agent.hostname for the life of 7.x but mark it as deprecated ASAP, then remove agent.hostname in 8.0.

I think agent.hostname should be replaced by host.hostname, however host.hostname is only present when add_host_metadata is configured for the events.

So I think we should also make Beats populate host.hostname regardless of add_host_metadata, just like host.name. We could instead deprecate it in favour of host.name, but the semantics are different: host.name is meant to be customizable (even though it defaults to host.hostname) whereas host.hostname is meant to be the "real" value on that host, just like agent.hostname is.

MikePaquette commented 5 years ago

@webmat we could mark agent.hostname as deprecated, but when it is removed, it will again break users current dashboards. Not sure how fast to push it.

Yes, having beats always populate host.hostname as a direct replacement for agent.hostname regardless of whether add_host_metadata is enabled is indeed an option. Let's decide on one approach. In fact, add_host_metadata is still listed as beta.

botelastic[bot] commented 4 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.