Security-Onion-Solutions / securityonion

Security Onion is a free and open platform for threat hunting, enterprise security monitoring, and log management. It includes our own interfaces for alerting, dashboards, hunting, PCAP, detections, and case management. It also includes other tools such as osquery, CyberChef, Elasticsearch, Logstash, Kibana, Suricata, and Zeek.
https://securityonion.net
3.22k stars 498 forks source link

Suricata and Grafana performance changes #803

Closed dougburks closed 1 year ago

dougburks commented 4 years ago

Per https://suricata.readthedocs.io/en/latest/performance/tuning-considerations.html, we should set defaults and allow tuning for high performance.

In suricata.yaml, we should increase the default max-pending-packets value to at least what we set in current platform, which is 5000 and allow it to be tuned.

We should probably enable the following under af-packet:

    #use-mmap: yes
    #tpacket-v3: yes

We should expose the following so the user can increase when necessary:

    #ring-size: 2048

Grafana should display additional stats from Suricata's stats.log so that we can tell if any memcap limitations are being reached and need to be tuned. It would be nice if Grafana could show the value from suricata.yaml and compare that to the actual usage in stats.log.

TOoSmOotH commented 4 years ago

Suricata config items are complete. Just need grafana

dougburks commented 4 years ago

Looks like tpacket-v3 is still commented out:

https://github.com/Security-Onion-Solutions/securityonion-saltstack/blob/dev/salt/suricata/files/suricata.yaml#L628

https://github.com/Security-Onion-Solutions/securityonion-saltstack/blob/dev/salt/suricata/files/suricataMETA.yaml#L628

Is that intentional?

UPDATE: Mike pointed out that tpacket-v3 is enabled here: https://github.com/Security-Onion-Solutions/securityonion-saltstack/blob/feature/suripillar/salt/suricata/files/defaults.yaml#L767

TOoSmOotH commented 4 years ago

We need to pull the metrics out and send them to telegraf and create additional graphs. Graph memuse. Max the graph at the current memcap.