opnsense / core

OPNsense GUI, API and systems backend
https://opnsense.org/
BSD 2-Clause "Simplified" License
3.37k stars 757 forks source link

[feature request] Suricata - Add option for eve-log over syslog #3401

Closed Benzy-Louis closed 5 years ago

Benzy-Louis commented 5 years ago

Hello,

I think it's not already asked (I did not find this specific feature request).

Is it possible to add the option to send the "eve-log" format over syslog as an alternative to the default file option which is set in the configuration template?

 - eve-log:
      enabled: yes
      type: syslog #file|syslog|unix_dgram|unix_stream
      #filename: eve.json
      # the following are valid when type: syslog above
      identity: "suricata"
      facility: local5
      level: Info  ## possible levels: Emergency, Alert, Critical,
                   ## Error, Warning, Notice, Info, Debug

This will be useful in a use case where the syslog is further processed over a logstash or graylog server to produce dashboards. Thanks,

Ruben.

AdSchellevis commented 5 years ago

It has been asked before, although I can't find the issue right away (it could also have been on the forum).

But since suricata only supports one eve.log output, it would kill the features inside OPNsense that depend on the eve.json file being there.

A lot of people use filebeat to ship these logs, maybe that's an option for your use case as well.

Benzy-Louis commented 5 years ago

Hello thanks for your answer. I know about the filebeat package but for my use case I'd need to be able to do that without having to use the shell (for example to install/configure filebeat). Which features depend on the eve.json file being there?

AdSchellevis commented 5 years ago

the alert view in the interface and the et-pro telemetry package for example also assumes the log file is there.

Benzy-Louis commented 5 years ago

Ok.

Benzy-Louis commented 5 years ago

According to suricata's documentation one can have multiple outputs for the eve-log, https://github.com/OISF/suricata/blob/d6903e70c1b653984ca95f8808755efbc6a9ece4/doc/userguide/output/eve/eve-json-output.rst#multiple-logger-instances, it's only for the drop logs that there can only be one output, I've open a feature request on the suricata project related to that https://redmine.openinfosecfoundation.org/issues/2935.

Another problem that I see is that the remote-syslog output is truncated after 1024 characters (1 KB) and at times the full message with the eve-log json spans more than 1024 characters, can this be tweaked somewhere or it's the limit of the syslog output ?

AdSchellevis commented 5 years ago

If we can have overlapping log targets, I don't mind looking into to adding a feature.

About the syslog length, I'm afraid this is a limitation of udp syslog (https://tools.ietf.org/html/rfc3164). Newer syslog frameworks support other transports for larger messages.

AdSchellevis commented 5 years ago

For this feature to be useful, we first need to replace syslog with syslog-ng. There have been talks about it for quite some time, we might schedule this for 20.1

AdSchellevis commented 5 years ago

The syslog-ng code with tcp syslog support is available in our master branch and scheduled for release in 19.7.

For reference, IDS/IPS will gain an option called "Enable eve syslog output" : image

Which can then be intercepted in the new syslog-ng target menu

image

Using the following settings

image

I tried to experiment a bit with facility / identity in the suricata.yaml as well, but for some reason these don't seem to combine very well over the different syslog sections.

thedeadliestcatch commented 8 months ago

I do not think syslog inputs support Suricata format in Security Onion/ELK. Just came across this issue, but the usual recommended way is to leverage filebeat.