elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
42 stars 451 forks source link

[Journald] `host.hostname` is overwritten by the default `add_host_metadata` processor run in Filebeat #11717

Open belimawr opened 2 weeks ago

belimawr commented 2 weeks ago

When using the journald input from Filebeat (currently used by the "Custom Journald logs" and the System integration once https://github.com/elastic/integrations/pull/11618), the field host.hostname can be overwritten by the add_host_metadata processor that runs by default on Beats running under Elastic-Agent.

The default processors added by the Elastic-Agent (and not shown in the policy/Fleet UI are):

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

This is added by this piece of code from Beats when running under Elastic-Agent:

func defaultProcessors() []mapstr.M {
    // processors:
    // - add_host_metadata:
    //  when.not.contains.tags: forwarded
    // - add_cloud_metadata: ~
    // - add_docker_metadata: ~
    // - add_kubernetes_metadata: ~

    return []mapstr.M{
        {
            "add_host_metadata": mapstr.M{
                "when.not.contains.tags": "forwarded",
            },
        },
        {"add_cloud_metadata": nil},
        {"add_docker_metadata": nil},
        {"add_kubernetes_metadata": nil},
    }
}

That is later called during startup.

One way to avoid host.hostname being overwritten is to add the forwarded tag to the events produced by the journald input (or any input).

Currently the forwarded tag is not added by default.

elasticmachine commented 2 weeks ago

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)