Closed mmguero closed 3 months ago
Here's a recap of the changes for this:
suricata_config_populate.py
that take effect only for live capture (i.e., when SURICATA_LIVE_CAPTURE
is true
):
SURICATA_EVE_FILENAME_PATTERN
- how live eve.json files are named (default eve-%Y%m%d_%H%M%S.json
)SURICATA_EVE_THREADED
- controls threaded file output (default false
)SURICATA_EVE_ROTATE_INTERVAL
- controls eve.json file rotation (default 1h
)eve-clean-logs.sh
from the Suricata containers for cleaning old eve.json logs. This is now handled by the filebeat
container by the clean-processed-folder.py
script in the same way the Zeek logs are, with behavior controlled by LOG_CLEANUP_MINUTES
env variable for Suricata logs.LOG_CLEANUP_MINUTES
and ZIP_CLEANUP_MINUTES
from upload-common.env
to filebeat.env
as filebeat
is the only container that uses those.filebeat.yml
configuration files on Malcolm and hedgehog to look for eve*.json
as the matching wildcard for harvesting.filebeat-clean-zeeklogs-processed-folder.py
to clean-processed-folder.py
as it is not used for more than just Zeek scripts. Added logic in there differentiating between Zeek and Suricata logs, and check both the suricata and suricata-live folders for old logs to prune. As the suricata documentation reads,
We need to look at adding to suricata_config_populate.py the ability to generate suricata.yaml such that suricata's eve.json is split/rotated so it doesn't grow as large.
There are a few things we could do here as possibilities
rotate-interval: hour
)types
in the example below)