elastic / beats

:tropical_fish: Beats - Lightweight shippers for Elasticsearch & Logstash
https://www.elastic.co/products/beats
Other
108 stars 4.93k forks source link

[8.x](backport #41555) Journald support for System module #41670

Closed mergify[bot] closed 3 days ago

mergify[bot] commented 3 days ago

Proposed commit message

This commit adds journald support for the System module, both filesets now have a use_journald variable that can be set to force using Journald to ingest syslog and auth logs.

The ingest pipelines are updated, now there is an entrypoint pipeline that selects the correct one according to the field input.type.

System tests are also added.

Checklist

## Disruptive User Impact

Author's Checklist

How to test this PR locally

  1. Package Filebeat from this PR
  2. Configure the ES output and Kibana credentials
  3. Enable the system module (./filebeat modules enable system), set var.use_journald: true for both filesets (edit modules.d/system.yml)
  4. Setup the assets: ./filebeat setup --modules system (this requires Kibana credentials correctly set)
  5. Run Filebeat as root
  6. Look at the logs in the filebeat-* data view, filter by event.dataset: system.syslog or event.dataset: system.auth, ensure the logs are correctly ingested
  7. Look at the system module dashboards, ensure they're working/show data.

Related issues

## Use cases

Screenshots

Dashboards, journald and logs side by side

![2024-11-08_10-54](https://github.com/user-attachments/assets/fa0c746a-1331-4ad1-86cd-bae0ec4d18fd) ![2024-11-08_10-54_1](https://github.com/user-attachments/assets/7eb221ee-d6f9-441f-938a-84d8d8118591) ![2024-11-08_10-55](https://github.com/user-attachments/assets/b0b8a554-8140-43f9-af14-4e18b1e3152a) ![2024-11-08_10-55_1](https://github.com/user-attachments/assets/b4f96dab-c634-4525-9d32-cba4f859b389)

Events, journald and logs side by side

![2024-11-08_10-59](https://github.com/user-attachments/assets/986e4f0e-4c1b-4672-9e58-f6fa22a199bd)

## Logs


This is an automatic backport of pull request #41555 done by Mergify.

botelastic[bot] commented 3 days ago

This pull request doesn't have a Team:<team> label.