Closed DigiAngel closed 3 years ago
Anyone?
Apologies, not ignoring you. I just haven't had the time to work on this project at all the last few weeks. One of us will get to it when we are able.
Appreciate it....I realize Defcon and BH take up everyone's time :)
What does your /opt/snort/etc/rules/snort-pulledpork.rules
file look like?
It's beefy...looks solid...about 8846 active rules by the look of it...4.9 meg size.
When you run Snort are you including the SO rule path as suggested in the comments at the top of the file?
Yes....it's the start of this ticket :)
So it is... drinks more coffee
Welcome to my world!
I can get this back to working state by manually copying files and making a couple tweaks to the included rule paths with a fresh lightspeed package if that would help? This is just on a test box so ya..anything goes really.
I can get this back to working state by manually copying files and making a couple tweaks to the included rule paths with a fresh lightspeed package if that would help? This is just on a test box so ya..anything goes really.
Yes, please. That would be a great help.
Ok....done....lint's fine..will include full output at the end. Here's the setup:
ips =
{
enable_builtin_rules = true,
rules = [[
include /opt/snort/etc/rules/includes.rules
include /opt/snort/etc/local.rules
include /opt/snort/etc/rules/so_rules/includes.rules
]],
variables = default_variables
}
in the lightspd dir:
sudo cp -R builtins/* /opt/snort/etc/rules/builtins
sudo cp -R modules/3.1.9.0/ubuntu-x64/so_rules/ /opt/snort/etc/rules/
sudo cp -R modules/stubs/* /opt/snort/etc/rules/so_rules/
sudo cp -R rules/*/*.rules /opt/snort/etc/rules/
here's a find within the rules dir...snips are where consecutive rules files were:
./snort3-file-office.rules
./snort3-deleted.rules
<snip>
./snort3-server-mail.rules
./snort3-server-other.rules
./includes.rules
./snort3-file-other.rules
./snort3-os-linux.rules
<snip>
./snort3-content-replace.rules
./snort3-policy-other.rules
./snort3-browser-plugins.rules
./snort3-pua-adware.rules
./builtins
./builtins/rulestates-security-ips.states
./builtins/builtins.rules
./builtins/rulestates-max-detect-ips.states
./builtins/rulestates-balanced-ips.states
./builtins/rulestates-no-rules-active.states
./builtins/rulestates-connectivity-ips.states
./snort3-pua-other.rules
./snort3-server-samba.rules
./snort3-malware-cnc.rules
./snort3-os-solaris.rules
./snort3-app-detect.rules
./snort3-browser-firefox.rules
./snort3-protocol-imap.rules
./snort3-protocol-icmp.rules
./snort3-file-multimedia.rules
./snort3-browser-ie.rules
./snort3-pua-toolbars.rules
./snort3-file-flash.rules
./so_rules
./so_rules/file-flash.rules
./so_rules/browser-ie.so
./so_rules/file-image.so
./so_rules/os-windows.rules
./so_rules/indicator-shellcode.so
<snip>
./so_rules/browser-chrome.so
./so_rules/protocol-tftp.rules
./so_rules/malware-cnc.rules
./so_rules/exploit-kit.so
./so_rules/os-other.rules
./so_rules/rulestates-connectivity-ips.states
./so_rules/server-webapp.rules
./snort3-file-identify.rules
lint
08:41:12 me@cape:/opt/snort/etc$] sudo /opt/snort/bin/snort --daq-dir=/opt/snort/libdaq/lib/daq --plugin-path=/opt/snort/etc/rules/so_rules -c /opt/snort/etc/snort.lua -T
--------------------------------------------------
o")~ Snort++ 3.1.9.0
--------------------------------------------------
Loading /opt/snort/etc/snort.lua:
Loading /opt/snort/etc/snort_defaults.lua:
Finished /opt/snort/etc/snort_defaults.lua:
Loading /opt/snort/etc/file_magic.lua:
Finished /opt/snort/etc/file_magic.lua:
Loading /opt/snort/etc/threshold.conf:
Finished /opt/snort/etc/threshold.conf:
alert_syslog
ssh
hosts
host_cache
pop
so_proxy
stream_tcp
smtp
packets
dce_http_proxy
stream_icmp
normalizer
stream_udp
alerts
search_engine
alert_fast
suppress
ips
dce_smb
binder
wizard
file_id
port_scan
dce_http_server
dce_tcp
ssl
sip
network
http2_inspect
host_tracker
http_inspect
stream_user
stream_ip
trace
classifications
active
log_pcap
decode
daq
stream
references
output
process
dns
dce_udp
imap
stream_file
Finished /opt/snort/etc/snort.lua:
Loading ips.rules:
Loading /opt/snort/etc/rules/includes.rules:
Loading snort3-app-detect.rules:
Finished snort3-app-detect.rules:
Loading snort3-browser-chrome.rules:
Finished snort3-browser-chrome.rules:
Loading snort3-browser-firefox.rules:
Finished snort3-browser-firefox.rules:
Loading snort3-browser-ie.rules:
Finished snort3-browser-ie.rules:
Loading snort3-browser-other.rules:
Finished snort3-browser-other.rules:
Loading snort3-browser-plugins.rules:
Finished snort3-browser-plugins.rules:
Loading snort3-browser-webkit.rules:
Finished snort3-browser-webkit.rules:
Loading snort3-content-replace.rules:
Finished snort3-content-replace.rules:
Loading snort3-exploit-kit.rules:
Finished snort3-exploit-kit.rules:
Loading snort3-file-executable.rules:
Finished snort3-file-executable.rules:
Loading snort3-file-flash.rules:
Finished snort3-file-flash.rules:
Loading snort3-file-identify.rules:
Finished snort3-file-identify.rules:
Loading snort3-file-image.rules:
Finished snort3-file-image.rules:
Loading snort3-file-java.rules:
Finished snort3-file-java.rules:
Loading snort3-file-multimedia.rules:
Finished snort3-file-multimedia.rules:
Loading snort3-file-office.rules:
Finished snort3-file-office.rules:
Loading snort3-file-other.rules:
Finished snort3-file-other.rules:
Loading snort3-file-pdf.rules:
Finished snort3-file-pdf.rules:
Loading snort3-indicator-compromise.rules:
Finished snort3-indicator-compromise.rules:
Loading snort3-indicator-obfuscation.rules:
Finished snort3-indicator-obfuscation.rules:
Loading snort3-indicator-scan.rules:
Finished snort3-indicator-scan.rules:
Loading snort3-indicator-shellcode.rules:
Finished snort3-indicator-shellcode.rules:
Loading snort3-malware-backdoor.rules:
Finished snort3-malware-backdoor.rules:
Loading snort3-malware-cnc.rules:
Finished snort3-malware-cnc.rules:
Loading snort3-malware-other.rules:
Finished snort3-malware-other.rules:
Loading snort3-malware-tools.rules:
Finished snort3-malware-tools.rules:
Loading snort3-netbios.rules:
Finished snort3-netbios.rules:
Loading snort3-os-linux.rules:
Finished snort3-os-linux.rules:
Loading snort3-os-mobile.rules:
Finished snort3-os-mobile.rules:
Loading snort3-os-other.rules:
Finished snort3-os-other.rules:
Loading snort3-os-solaris.rules:
Finished snort3-os-solaris.rules:
Loading snort3-os-windows.rules:
Finished snort3-os-windows.rules:
Loading snort3-policy-multimedia.rules:
Finished snort3-policy-multimedia.rules:
Loading snort3-policy-other.rules:
Finished snort3-policy-other.rules:
Loading snort3-policy-social.rules:
Finished snort3-policy-social.rules:
Loading snort3-policy-spam.rules:
Finished snort3-policy-spam.rules:
Loading snort3-protocol-dns.rules:
Finished snort3-protocol-dns.rules:
Loading snort3-protocol-finger.rules:
Finished snort3-protocol-finger.rules:
Loading snort3-protocol-ftp.rules:
Finished snort3-protocol-ftp.rules:
Loading snort3-protocol-icmp.rules:
Finished snort3-protocol-icmp.rules:
Loading snort3-protocol-imap.rules:
Finished snort3-protocol-imap.rules:
Loading snort3-protocol-nntp.rules:
Finished snort3-protocol-nntp.rules:
Loading snort3-protocol-other.rules:
Finished snort3-protocol-other.rules:
Loading snort3-protocol-pop.rules:
Finished snort3-protocol-pop.rules:
Loading snort3-protocol-rpc.rules:
Finished snort3-protocol-rpc.rules:
Loading snort3-protocol-scada.rules:
Finished snort3-protocol-scada.rules:
Loading snort3-protocol-services.rules:
Finished snort3-protocol-services.rules:
Loading snort3-protocol-snmp.rules:
Finished snort3-protocol-snmp.rules:
Loading snort3-protocol-telnet.rules:
Finished snort3-protocol-telnet.rules:
Loading snort3-protocol-tftp.rules:
Finished snort3-protocol-tftp.rules:
Loading snort3-protocol-voip.rules:
Finished snort3-protocol-voip.rules:
Loading snort3-pua-adware.rules:
Finished snort3-pua-adware.rules:
Loading snort3-pua-other.rules:
Finished snort3-pua-other.rules:
Loading snort3-pua-p2p.rules:
Finished snort3-pua-p2p.rules:
Loading snort3-pua-toolbars.rules:
Finished snort3-pua-toolbars.rules:
Loading snort3-server-apache.rules:
Finished snort3-server-apache.rules:
Loading snort3-server-iis.rules:
Finished snort3-server-iis.rules:
Loading snort3-server-mail.rules:
Finished snort3-server-mail.rules:
Loading snort3-server-mssql.rules:
Finished snort3-server-mssql.rules:
Loading snort3-server-mysql.rules:
Finished snort3-server-mysql.rules:
Loading snort3-server-oracle.rules:
Finished snort3-server-oracle.rules:
Loading snort3-server-other.rules:
Finished snort3-server-other.rules:
Loading snort3-server-samba.rules:
Finished snort3-server-samba.rules:
Loading snort3-server-webapp.rules:
Finished snort3-server-webapp.rules:
Loading snort3-sql.rules:
Finished snort3-sql.rules:
Loading snort3-x11.rules:
Finished snort3-x11.rules:
Finished /opt/snort/etc/rules/includes.rules:
Loading /opt/snort/etc/local.rules:
Finished /opt/snort/etc/local.rules:
Loading /opt/snort/etc/rules/so_rules/includes.rules:
Loading browser-chrome.rules:
Finished browser-chrome.rules:
Loading browser-ie.rules:
Finished browser-ie.rules:
Loading browser-other.rules:
Finished browser-other.rules:
Loading browser-webkit.rules:
Finished browser-webkit.rules:
Loading exploit-kit.rules:
Finished exploit-kit.rules:
Loading file-executable.rules:
Finished file-executable.rules:
Loading file-flash.rules:
Finished file-flash.rules:
Loading file-image.rules:
Finished file-image.rules:
Loading file-java.rules:
Finished file-java.rules:
Loading file-multimedia.rules:
Finished file-multimedia.rules:
Loading file-office.rules:
Finished file-office.rules:
Loading file-other.rules:
Finished file-other.rules:
Loading file-pdf.rules:
Finished file-pdf.rules:
Loading indicator-shellcode.rules:
Finished indicator-shellcode.rules:
Loading malware-cnc.rules:
Finished malware-cnc.rules:
Loading malware-other.rules:
Finished malware-other.rules:
Loading netbios.rules:
Finished netbios.rules:
Loading os-linux.rules:
Finished os-linux.rules:
Loading os-other.rules:
Finished os-other.rules:
Loading os-windows.rules:
Finished os-windows.rules:
Loading policy-other.rules:
Finished policy-other.rules:
Loading policy-social.rules:
Finished policy-social.rules:
Loading protocol-dns.rules:
Finished protocol-dns.rules:
Loading protocol-other.rules:
Finished protocol-other.rules:
Loading protocol-scada.rules:
Finished protocol-scada.rules:
Loading protocol-snmp.rules:
Finished protocol-snmp.rules:
Loading protocol-tftp.rules:
Finished protocol-tftp.rules:
Loading protocol-voip.rules:
Finished protocol-voip.rules:
Loading server-iis.rules:
Finished server-iis.rules:
Loading server-mail.rules:
Finished server-mail.rules:
Loading server-mysql.rules:
Finished server-mysql.rules:
Loading server-oracle.rules:
Finished server-oracle.rules:
Loading server-other.rules:
Finished server-other.rules:
Loading server-webapp.rules:
Finished server-webapp.rules:
Finished /opt/snort/etc/rules/so_rules/includes.rules:
Finished ips.rules:
--------------------------------------------------
rule counts
total rules loaded: 45436
text rules: 42081
builtin rules: 589
so rules: 2766
option chains: 45436
chain headers: 1820
--------------------------------------------------
port rule counts
tcp udp icmp ip
any 2556 427 490 310
src 14722 168 0 0
dst 26328 1285 0 0
both 110 73 0 0
total 43716 1953 490 310
--------------------------------------------------
ips policies rule stats
id loaded shared enabled file
0 45436 0 45436 /opt/snort/etc/snort.lua
--------------------------------------------------
flowbits
defined: 703
not checked: 65
--------------------------------------------------
service rule counts to-srv to-cli
bgp: 3 0
dcerpc: 108 21
dhcp: 49 20
dns: 283 114
drda: 5 0
file: 48 48
ftp: 199 23
ftp-data: 138 10389
gopher: 0 1
http: 13238 13709
http2: 13238 13709
ident: 1 0
igmp: 1 1
imap: 176 10635
ipp: 1 0
irc: 40 14
ircd: 9 3
java_rmi: 52 2
kerberos: 36 6
ldap: 42 8
ldp: 1 0
mdns: 12 11
mysql: 66 6
netbios-dgm: 11 11
netbios-ns: 15 5
netbios-ssn: 567 135
netware: 2 0
nntp: 2 2
ntp: 37 8
openvpn: 16 16
pop3: 136 10638
postgresql: 6 0
printer: 6 0
radius: 4 3
rdp: 11 17
rtmp: 1 4
rtp: 2 2
rtsp: 18 2
sip: 371 47
smtp: 9577 76
snmp: 80 18
ssdp: 13 0
ssh: 5 2
ssl: 196 207
sunrpc: 116 10
syslog: 4 0
teamview: 1 2
telnet: 64 18
tftp: 22 6
vnc: 4 1
vnc-server: 11 4
wins: 3 0
total: 39047 59954
--------------------------------------------------
fast pattern port groups src dst any
packet: 284 996 4
--------------------------------------------------
fast pattern service groups to-srv to-cli
packet: 49 39
key: 6 2
header: 10 10
body: 8 0
file: 12 8
raw_key: 2 0
raw_header: 2 0
method: 2 0
stat_code: 0 2
stat_msg: 0 2
cookie: 2 0
--------------------------------------------------
search engine
--------------------------------------------------
pcap DAQ configured to passive.
Snort successfully validated the configuration (with 0 warnings).
o")~ Snort exiting
Any movement on this?
Don't think we've had any time to make movement on this, no.
Completely understand..thank you.
I'm looking over this ticket now, and it looks like you're doing everything correctly. The error you're seeing tells me that when snort loads the single text rules file, it's encountering a number of rules that are .so rules, but for some reason it can't locate the necessary .so files, even though it looks like you have included them properly, and they exist in the folder.
I'm spooling up an ubuntu vm to test this on, since I can't seem to think of anything from what you've posted that could be causing this. If possible: can you run PP3 with the -vv flag (very verbose output). You can use tee or redirect the stdout to save the output to a text file and attach it to this ticket? PP3 should remove your oinkcode from the output automatically. You might also try deleting the contents of the so_rules folder before running PP3, we have an open ticket to add that functionality, but it hasn't been done yet, and (long shot) might be part of the problem.
I'll also assume you're running PP3 with elevated rights, and you're using the latest version of PP3 (last updated on July 24th)
Ok, I've re-created the issue on my system. I can't guarantee you have the same problem, but I suspect this is it. This is a bug (def. not a feature). The problem looks to be the way determines which folder to pull the .so file from (they're versioned). It should be looking for a folder version that is equal to or less than the actual version of snort running, but for some reason that's not working (I'll dig into the code, but I don't suspect a complicated fix, probably need to flip a greater than to a less than). Here's how I identified the issue, from the PP3 output (line 4 below gives it away):
Processing json manifest file /tmp/PulledPork-2021.08.30-14.28.11/extracted_rulesets/Talos_LightSPD/lightspd/manifest.json
Found 11 versions of snort in the manifest file: ['3.1.9.0-0', '3.1.8.0-0', '3.1.7.1-73', '3.1.7.0-0', '3.1.10.0-0', '3.1.1.0-0', '3.1.0.1-174', '3.1.0.1-149', '3.1.0.0-0', '3.0.3-4', '3.0.3-1']
Looking for a version in the manifest file that is less than or equal to our current snort Version: 3.1.6.0
Using snort version 3.1.10.0-0 from lightSPD manifest file. Actual Snort version is: 3.1.6.0
policies_path from lightSPD Manifest file for snort 3.1.10.0-0 is: policies/3.1.0.1-174/
modules_path from lightSPD Manifest file for snort 3.1.10.0-0 is: modules/3.1.10.0/ubuntu-x64/
I'll track this down and let you know when it's fixed. If you copy the .so rules files from the correct folder in the LightSPD archive to /opt/snort/etc/rules/so_rules, it should not throw those errors. Noah
Ok cool thanks. So originally I wasn't going to use pp as now that we don't have to dump stub rules it was really just a copy paste. It was when I started seeing different so rules directories that things got interesting. I know that the LightSPD package is designed for Firepower and what not...but maybe you could suggest making some symlinks? Say....if I'm running 3.1.8.0, but the so rules in 3.1.9.0 will work fine with them, then just have the package maintainer symlink them? At that point, for a pp and a manual copy paste perspective, it's simply a one for one when you script it....i.e. find the version of snort running/binary, and copy the correct version so_rules directory over and call it good. Just a thought :) Still need that pp -vv run?
Ok, this is slightly more challenging to solve due to the funky version numbers that occur in the LightSPD package. For example, here's what I see from the latest: ['3.1.9.0-0', '3.1.8.0-0', '3.1.7.1-73', '3.1.7.0-0', '3.1.10.0-0', '3.1.1.0-0', '3.1.0.1-174', '3.1.0.1-149', '3.1.0.0-0', '3.0.3-4', '3.0.3-1']
if those existed without the hyphen, this would be simple (one liner). however you see some of those early versions (3.0.3-4) dont' really line up. What i'll probably do is swap the hyphen with a dot, and for those two early ones that only have n.n.n-x, I'll modify it to n.n.n.0-x. That'll allow for an easy sort.
Pimpy....can't wait to see the results :)
@DigiAngel : Don't need the -vv yet, I found a bug and i'm fixgin it. regarding symlinks: they aren't needed. The idea is that you find the folder version that matches your version of snort, and if there isn't an exact match, you find the folder with the most recent version below your version number. PP will determine your version of Snort (parses the output of snort -v if you don't specify the version number in your PP.conf), and then it looks for the correct .so rules folder, based on that version number. What you're suggesting is what we do (when it works).
LoL...awesome....happy to be part of the process :)
This should be fixed now. Solution wasn't as clean as I would have liked, due to different numbering schemes in the LightSPD file, but after normalization of the numbering it looks good. Please test with the latest version from git.
you have to include the path to the so_rules using --plugin-path $Path when running snort
@Crono-dev is right, just do for example: /usr/local/bin/snort -c /usr/local/etc/snort/snort.lua --plugin-path /usr/local/etc/so_rules/ -s 65535 -k none -l /var/log/snort -i eth0 -m0x1b
if you installed from documentation on Ubuntu I had same issue like you showed.
Lint line:
pulledpork.conf:
ips entry in snort.lua:
/opt/snort/etc/rules/so_rules dir:
errors:
19 through 265 in the snort-pulledpork.rules are all the soid rules.