search

jasonish / py-idstools

idstools: Snort and Suricata Rule and Event Utilities in Python (Including a Rule Update Tool)
Other
277 stars 85 forks source link

Feature request: snort.org support in rulecat #52

Closed sevdog closed 7 years ago

sevdog commented 7 years ago

This request is divided in two step:

1. Snort community rules are not handled properly:

$ idstools-rulecat --url "https://www.snort.org/rules/community" --merged community.rules
2017-05-16 17:51:56,704 - <INFO> -- Found Suricata version 3.1.2 at /usr/bin/suricata.
2017-05-16 17:51:56,705 - <INFO> -- Checking https://www.snort.org/rules/community.md5.
2017-05-16 17:51:57,674 - <ERROR> -- Failed to check remote checksum: HTTP Error 406: Not Acceptable
2017-05-16 17:51:57,675 - <INFO> -- Fetching https://www.snort.org/rules/community.
 100% - 295175/295175                 
2017-05-16 17:51:59,439 - <INFO> -- Done.
2017-05-16 17:51:59,439 - <INFO> -- Loaded 0 rules.
2017-05-16 17:51:59,440 - <INFO> -- Disabled 0 rules.
2017-05-16 17:51:59,440 - <INFO> -- Enabled 0 rules.
2017-05-16 17:51:59,440 - <INFO> -- Modified 0 rules.
2017-05-16 17:51:59,440 - <INFO> -- Dropped 0 rules.
2017-05-16 17:51:59,440 - <INFO> -- Enabled 0 rules for flowbit dependencies.
2017-05-16 17:51:59,440 - <INFO> -- Writing community.rules: added: 0; removed 0; modified: 0

But if I manually download the file from my browser what I get is a .tar archive which contains a community.rules with about 3500 rules.

2. Add parameters like ET also for snort

With the latest version (master) also snort registered rules download is supperted via the --url parameter:

idstools-rulecat --url "https://www.snort.org/rules/snortrules-snapshot-2976.tar.gz?oinkcode=<onikcode>"

What I propose is to add a couple of arguments to handle snort like ET:

--snort-version                 Snort version (choose from 2976, 2983, 2990)
--snort-community               Download snort community rules
--snort-registered <oink-code>  Download snort registered rules providing your oink-code

The URLs for snort rules are:

# snort community
https://www.snort.org/rules/community

# snort registered
https://www.snort.org/rules/snortrules-snapshot-<snort-version>.tar.gz?oinkcode=<oinkcode>
marcindulak commented 7 years ago

With an alternative community rules download url used by pulledpork https://github.com/shirkdog/pulledpork/blob/a2c1b6772f7dfd178a54aa2f4e9f4f04a9072389/etc/pulledpork.conf#L21, this works for me with https://github.com/jasonish/py-idstools/releases/tag/0.6.0 (only with Python2, due to https://github.com/jasonish/py-idstools/issues/53)

idstools-rulecat --url "https://snort.org/downloads/community/community-rules.tar.gz" --merged community.rules
idstools-rulecat --url "https://www.snort.org/rules/snortrules-snapshot-2983.tar.gz?oinkcode=<oinkcode>" --merged snort.rules
jasonish commented 7 years ago

I've updated git master to handle the case 1 - even though I still suggest linking directly to the full filename, I haven't seen that shorter URL before.

There are some other issues I think I need to deal with before adding proper support for the Snort URLs:

Pulled Pork is really the tool to use for Snort.

sevdog commented 7 years ago

It's ok if you want to keep the two enviroment separeted.

Btw, the URL of case 1 is found on official snort site: image

You can close the Issue if support for Snort/Talos Rules is not intended.

Thanks.

jasonish commented 7 years ago

Ah, I believe there is example is wrong as well. That will download the file as just "community" (missing extension) so the second command will fail for cut and pasters. I've notified them.

Closing for now. Snort support is something thats always on my mind, so feel free to raise this again sometime in the near future if I haven't myself.