elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
12.06k stars 4.89k forks source link

[filebeat] exclude_lines dont work with docker or kubernetes doc examples (input type container) #40049

Open ebuildy opened 1 week ago

ebuildy commented 1 week ago

From documentation at https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover-hints.html#_co_elastic_logsexclude_lines

Capture d’écran, le 2024-06-27 à 21 56 41

This say we can use exclude_lines input option with Docker or Kubernetes, but this dont work.

When reading code source, I see excludeLines only for type: filestream not for Docker or Log: https://github.com/elastic/beats/blob/main/filebeat/input/filestream/input.go#L402.

I see container input is using log input under the wood (https://github.com/elastic/beats/blob/main/filebeat/input/container/input.go#L74), that is deprecated, do you plan to use filestream instead?


edit:

Related to https://github.com/elastic/beats/issues/34393

I see there is from 1 year a discussion around "replace type: container  by type: filestream", some official documentation is with filestream, and some with container :-/

type: filestream --> https://github.com/elastic/beats/blob/main/deploy/kubernetes/filebeat-kubernetes.yaml type: container --> https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html

So the big question, how configure filebeat with kubernetes ^^ ?

elasticmachine commented 4 days ago

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

VihasMakwana commented 4 days ago

@ebuildy exclude_lines also works for log input https://github.com/elastic/beats/blob/main/filebeat/input/log/harvester.go#L496C21-L513. Can your filebeat config and any logs, if possible?