StamusNetworks / scirius

Scirius is a web application for Suricata ruleset management and threat hunting.
GNU General Public License v3.0
635 stars 150 forks source link

Upload/URL pull of subscriber snort rules failes. #101

Open Sc0th opened 7 years ago

Sc0th commented 7 years ago

Attempting to upload the subscriber Snort rules (version 2983) results in '413 Request Entity Too Large' error being kicked back from nginx and the rules are not imported.

Sc0th commented 7 years ago

Ok, fixed this part by upping client_max_body_size to 100M, however now I get '504 Gateway Time-out' seems to be set at 60 seconds and cannot see where this is set, any pointers much appreciated...

regit commented 7 years ago

Hi @Sc0th,

How are you running scirius (fcgi, gunicorn) ?

Sc0th commented 7 years ago

Hi! - Excuse my ignorance, but I have no idea, I am a running SELKS, whilst I have a technical background, this is all very new to me, pretty much a default install from the latest 3.0 ISO...

regit commented 7 years ago

You should try to set

    fastcgi_read_timeout 300s;

below fastcgi_pass in /etc/nginx/sites-enabled/stamus.conf 

And then restart nginx.

Sc0th commented 7 years ago

Splendid, thank you, much closer! - now I see 'UNIQUE constraint failed: rules_rule.sid' - this looks more like a user error on my part, not sure this is the correct forum to discuss; I will mosey over to the other forums if I cannot find a fix.

regit commented 7 years ago

This means that you did already import a source containing similar rules. Maybe you have imported open ruleset from Snort or something alike. Try to delete the other source and reimport.

Sc0th commented 7 years ago

That makes sense, I did have the Snort community rules loaded, I have deleted them and rebooted for good measure, still seeing the same response. I will poke about a bit

Sc0th commented 7 years ago

If I delete all sources, I still get the same response, any chance there could be some leftovers from a previous failed attempt that are not getting cleaned up with a delete?

regit commented 7 years ago

It is really unlikely. Do you have information about the failling SID ?

Sc0th commented 7 years ago

I have just spent 30 mins trying to find an answer to that question, I have failed, no idea how to see what it's doing...

pevma commented 7 years ago

@Sc0th - what is the err that you are getting now?

Sc0th commented 7 years ago

Hi, this 'UNIQUE constraint failed: rules_rule.sid' in a red banner. This is when trying to upload snortrules-snapshot-2983.tar.gz, I have the SELKS default sources installed ET Open & SSLBL. The install is mostly untouched other than the above, a bit of network config & a couple of additional accounts.

Sc0th commented 7 years ago

Have removed all other sources and rulesets, still the same error

Sc0th commented 7 years ago

FWIW - The import of the Snort Community rules also now fails on the test phase, something here is rather borked, will blow it away and start again.

Sc0th commented 7 years ago

Sadly, after considerable faffing and numerous tries I have concluded it is actually not possible to import the Snort subscriber rule-set into the SELKS environment, this is a real shame, if you are reading this and need these rules, probably best to look elsewhere for the time being.

pevma commented 7 years ago

There are a couple of points here to consider:

trahtunberg commented 7 years ago

As I detected 2 files cause an error mentioned above UNIQUE constraint failed: rules_rule.sid:

  1. browser-ie.rules
  2. browser-plugins.rules

If you delete these 2 files from an archive - then it's uploaded normally.

Also you need:

  1. to add in /etc/nginx/nginx.conf the next line client_max_body_size 100M; in http { section
  2. to add fastcgi_read_timeout 300s; below fastcgi_pass in /etc/nginx/sites-enabled/stamus.conf
  3. Restart nginx with command: service nginx restart
pevma commented 7 years ago

@trahtunberg - thanks for the tip and sharing!

mrnerdhair commented 6 years ago

147 may help with this, though I'm still getting the same SID error.