idaholab / Malcolm

Malcolm is a powerful, easily deployable network traffic analysis tool suite for full packet capture artifacts (PCAP files), Zeek logs and Suricata alerts.
https://idaholab.github.io/Malcolm/
Other
327 stars 53 forks source link

adjustments to how Zeek intel files get generated among Malcolm's containers #456

Closed mmguero closed 2 months ago

mmguero commented 2 months ago

The issue is right now all of the Zeek containers attempt to generate the intel file, although it's shared amongst all of them.

This is exacerbated in a Kubernetes deployment where you scale up the number of zeek-live containers.

Here are the changes we need to make:

  1. Make the zeek-live containers not run the intel update at all. Just have them start up and start collecting, and in zeekdeploy.sh they'll notice updates to the intel files made externally to them and restart automatically. We need to not unilaterally remove the intel generation from zeek-deploy (since that's what the hedgehogs use as well) but will need to distinguish which of those two scenarios we're in.
  2. Make the zeek-offline container the only one in charge of doing the intel update. Whether this happens in the entrypoint (as is the case now) or cron-based is up for debate.
  3. move the working directory for creating the intel files to a temporary directory so that even if we did get two working on it at once they're not stepping on each others' toes
mmguero commented 2 months ago

in mmguero-dev/Malcolm@97ebeef, I did the following: