elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.17k stars 4.92k forks source link

Setting `publisher_pipeline.disable_host: true` has no effect #41046

Open chrisberkhout opened 3 weeks ago

chrisberkhout commented 3 weeks ago

The publisher_pipeline.disable_host option is documented as:

publisher_pipeline.disable_host

By default, all events contain host.name. This option can be set to true to disable the addition of this field to all events. The default value is false.

The setting is applied in libbeat/publisher/processing/default.go.

However, with publisher_pipeline.disable_host: true set, host.* fields will still be added, unless the forwarded tag is also set.

I suspect this is because there is a separate default processor to add host metadata when tags don't contain forwarded.

The documentation above, and some logic, refers only to host.name, but without the forwarded tag a range of fields will be set. For example:

    "host": {
      "hostname": "docker-fleet-agent",
      "os": {
        "kernel": "6.10.9-arch1-2",
        "codename": "focal",
        "name": "Ubuntu",
        "type": "linux",
        "family": "debian",
        "version": "20.04.6 LTS (Focal Fossa)",
        "platform": "ubuntu"
      },
      "containerized": false,
      "ip": [
        "172.18.0.7"
      ],
      "name": "docker-fleet-agent",
      "id": "d7b94aeb4f9141eaa5f345ec31e65c86",
      "mac": [
        "02-42-AC-12-00-07"
      ],
      "architecture": "x86_64"
    },

It would be good to remove the host.name and publisher_pipeline.disable_host: true logic and documentation, and document the host.* default processor behavior.

An integration that demonstrates the effect of these settings can be generated with this script.

elasticmachine commented 3 weeks ago

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)