search

opnsense / plugins

OPNsense plugin collection
https://opnsense.org/
BSD 2-Clause "Simplified" License
842 stars 635 forks source link

BE 22.4.1: os-intrusion-detection-content-snort-vrt 1.1_1: SC_ERR_NO_RULES_LOADED #3009

Closed Manfred-Knick closed 1 year ago

Manfred-Knick commented 2 years ago

Having run the "30 €" "private" snort VRT rulesets in former IPFipre installations, I was used to get multiple alerts a day.

Having migrated them to BE 22.4, now being upgraded to 22.4.1, being hit by none made me suspicious.

Resorting to "divide et impera", enabling only one source of rule sets at a time, gave the following results:

Thus one should not be too astonished that no Alerts are detected and reported ;-(

Stopping the sevice, and starting it again:

" Services: Intrusion Detection: Log File "

Date Severity Process Line

2022-06-14T18:01:36 Notice suricata
[100224] -- all 3 packet processing threads, 4 management threads initialized, engine started.

<-----> 2022-06-14T18:01:34 Warning suricata
[100224] -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 116 rule files specified, but no rules were loaded! <----->

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol http2 enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol http2 enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol rdp enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol mqtt enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol rfb enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Warning suricata [100123] -- [ERRCODE: SC_ERR_CONF_YAML_ERROR(242)] - App-Layer protocol sip enable status not set, so enabling by default. This behavior will change in Suricata 7, so please update your config. See ticket #4744 for more details.

2022-06-14T18:01:34 Notice suricata [100123] -- This is Suricata version 6.0.5 RELEASE running in SYSTEM mode

2022-06-14T18:01:24 Error suricata [100240] -- [ERRCODE: SC_ERR_SYSCALL(50)] - Unable to set caps for iface "em0": Invalid argument

2022-06-14T18:01:24 Notice suricata [100240] -- Stats for 'em0': pkts: 149291, drop: 0 (0.00%), invalid chksum: 111

2022-06-14T18:01:23 Notice suricata [100240] -- Stats for 'em1': pkts: 0, drop: 0 (nan%), invalid chksum: 0

2022-06-14T18:01:23 Notice suricata [100240] -- Stats for 'em2': pkts: 47662, drop: 0 (0.00%), invalid chksum: 0

2022-06-14T18:01:23 Notice suricata [100240] -- Signal Received. Stopping engine.

OPNsense-bot commented 2 years ago

Thank you for creating an issue. Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.

For more information about the policies for this repository, please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

The easiest option to gain traction is to close this ticket and open a new one using one of our templates.

AdSchellevis commented 2 years ago

best place to start looking is the general system log System: Log Files: General. The rule-updater.py should send feedback about downloads there.

Manfred-Knick commented 2 years ago

As mentioned in OP: "...all work fine, resulting into daily updates as expected"

Every day, rule-updater.py log entries confirm:

" | Notice | /rule-updater.py | download completed for https://www.snort.org/rules/snortrules-snapshot-31210.tar.gz?oinkcode=..."

AdSchellevis commented 2 years ago

Next question would be which files it actually did download, to list the rule files on disk:

ls -aslh /usr/local/etc/suricata/rules/
Manfred-Knick commented 2 years ago

Thanks for pointing me:

# ls -aslh /usr/local/etc/suricata/rules/ | wc -l 186

# ls -aslh /usr/local/etc/suricata/rules/ | grep snort | wc -l 116

But:

# ls -aslh /usr/local/etc/suricata/rules/ | grep snort | head -n 10 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.app-detect.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.attack-responses.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.backdoor.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.bad-traffic.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.blacklist.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.botnet-cnc.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.browser-chrome.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.browser-firefox.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.browser-ie.rules 0 -rw-r----- 1 root wheel 0B Jun 30 03:31 snort_vrt.browser-other.rules

Although assigned the correct download timestamp, all snort_vrt.* seem to be empty! That perfectly fits the error message above.

# df -h . Filesystem Size Used Avail Capacity Mounted on /dev/gpt/rootfs 136G 2.2G 123G 2% /

Manually downloading https://www.snort.org/rules/snortrules-snapshot-31210.tar.gz?oinkcode=$$$$$$$$ right now results into . . . . . snortrules-snapshot-31210.tar.gz. . . . . 10,0 MiB

extracting: sub-directries -> builtins, etc, rules, so_rules # llAR | grep rules

AdSchellevis commented 2 years ago

my assumption would be that the files are different in this version, the default seems to be 29151, I don't know if they still offer that, but comparing the contents of these files would probably make sense.

The definition expects files like rules/server-oracle.rules (https://github.com/opnsense/plugins/blob/master/security/intrusion-detection-content-snort-vrt/src/opnsense/scripts/suricata/metadata/rules/snort-vrt.xml)

Manfred-Knick commented 2 years ago

In the tar I find 66 files like snort3-*.rules: $ ls -A -R -1 | grep "snort3-" snort3-app-detect.rules snort3-browser-chrome.rules snort3-browser-firefox.rules snort3-browser-ie.rules snort3-browser-other.rules snort3-browser-plugins.rules snort3-browser-webkit.rules snort3-content-replace.rules snort3-exploit-kit.rules snort3-file-executable.rules snort3-file-flash.rules snort3-file-identify.rules snort3-file-image.rules snort3-file-java.rules snort3-file-multimedia.rules snort3-file-office.rules snort3-file-other.rules snort3-file-pdf.rules snort3-indicator-compromise.rules snort3-indicator-obfuscation.rules snort3-indicator-scan.rules snort3-indicator-shellcode.rules snort3-malware-backdoor.rules snort3-malware-cnc.rules snort3-malware-other.rules snort3-malware-tools.rules snort3-netbios.rules snort3-os-linux.rules snort3-os-mobile.rules snort3-os-other.rules snort3-os-solaris.rules snort3-os-windows.rules snort3-policy-multimedia.rules snort3-policy-other.rules snort3-policy-social.rules snort3-policy-spam.rules snort3-protocol-dns.rules snort3-protocol-finger.rules snort3-protocol-ftp.rules snort3-protocol-icmp.rules snort3-protocol-imap.rules snort3-protocol-nntp.rules snort3-protocol-other.rules snort3-protocol-pop.rules snort3-protocol-rpc.rules snort3-protocol-scada.rules snort3-protocol-services.rules snort3-protocol-snmp.rules snort3-protocol-telnet.rules snort3-protocol-tftp.rules snort3-protocol-voip.rules snort3-pua-adware.rules snort3-pua-other.rules snort3-pua-p2p.rules snort3-pua-toolbars.rules snort3-server-apache.rules snort3-server-iis.rules snort3-server-mail.rules snort3-server-mssql.rules snort3-server-mysql.rules snort3-server-oracle.rules snort3-server-other.rules snort3-server-samba.rules snort3-server-webapp.rules snort3-sql.rules snort3-x11.rules

Manfred-Knick commented 2 years ago

SNORT_from_tar.txt

AdSchellevis commented 2 years ago

and that's your issue likely, it should probably be a snort 2 file for this plugin.

Manfred-Knick commented 2 years ago

Reverting to latest "Subscription -> Snort v2.9" version:

. . . snortrules-snapshot-29200.tar.gz

2022-06-30T18:37:27 | Notice | /rule-updater.py | download completed for https://www.snort.org/rules/snortrules-snapshot-29200.tar.gz?oinkcode=$$$

ls -1 snort_vrt.* | wc -l 116

now being non-empty :-)

filling into "Rules" TAB entries :-)

Seems you were right! Will report back as soon as some of these rules have fired.

Thank you very much! Kind regards Manfred

Manfred-Knick commented 2 years ago

Proposal:

Plugin: os-intrusion-detection-content-snort-vrt :

https://github.com/opnsense/plugins/blob/master/security/intrusion-detection-content-snort-vrt/Makefile Enhance "Comment" with a tiny hint: - IDS Snort VRT ruleset (needs registration or subscription) + IDS Snort VRT 2.x ruleset (needs registration or subscription)

https://github.com/opnsense/plugins/blob/master/security/intrusion-detection-content-snort-vrt/src/opnsense/scripts/suricata/metadata/rules/snort-vrt.xml : Line 126 : - ... snortrules-snapshot-29151.tar.gz ... + ... snortrules-snapshot-29200.tar.gz ...

Manfred-Knick commented 2 years ago

Update:

Will report back as soon as some of these rules have fired.

Having run this for some weeks now (Hyperscan, on all "internal" but not the "Provider" interfaces, as suggested), I did get (very few) alerts from the ET Telemetry rule sets, but not a single one originating from the Snort VRT rule sets, which - in contrast to former IPFire behaviour - is irritating.

To me, the error entries in the Update logs just refer to errors in loading individual rules from . . . server-webapp.rules . . . server-other.rules . . . malware-cnc.rules . . . file-identify.rules suricata.log

Anything else I could provide? Kind regards

AdSchellevis commented 2 years ago

Usually it's about what's being measured, home networks versus non home networks and traffic already being dropped by the firewall in earlier stages. likely not a simple answer, the forum might be a better place to ask for help.

Manfred-Knick commented 1 year ago

ad@opnsense.org: Are here any plans for an update path to Snort 3 ? Kind regards Manfred

AdSchellevis commented 1 year ago

@Manfred-Knick not from my end, I don't think think the rules are compatible with suricata either to be honest

Manfred-Knick commented 1 year ago

Thanks a lot for your assessment !

OPNsense-bot commented 1 year ago

This issue has been automatically timed-out (after 180 days of inactivity).

For more information about the policies for this repository, please read https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md for further details.

If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.