search

xnih / satori

Python rewrite of passive OS fingerprinting tool
GNU General Public License v2.0
153 stars 25 forks source link

Log files contain too many dupes within short period of time #14

Closed xnih closed 3 years ago

xnih commented 3 years ago

So unlike windows satori version, where it would just update the timestamp, the python version spits out every instance and kills the log servers. Graylog with nxlog is only "smart" enough to dedupe them if they are within one second.

xnih commented 3 years ago

Added new feature "limit" that gives you the choice of how long to put another fingerprint out to screen.

python3 satori.py -i ens33 -r /home/xnih/Downloads/printer.pcap | wc -l 5016

python3 satori.py -i ens33 -r /home/xnih/Downloads/printer.pcap -l 1 | wc -l 214

So testing with one file dropped it from 5016 entries to screen down to 214. Time Limit is in minutes, could do 10, 60, or some other random #

To implement this it puts the fingerprint in a dictionary in the format: fingerprint, timestamp

I'm unsure how much memory this will take up as it builds up this dictionary and keeps it over time as the program runs, so this was implemented as an optional feature. I also know some may want to see "every" fingerprint to verify functionality.

Doing some initial testing before I commit later this week.

xnih commented 3 years ago

77 files from ctf at dc17 coming in at 7.3 Gb in size

./test1.sh Sun 12 Sep 2021 08:02:04 PM PDT 4965435 Sun 12 Sep 2021 09:21:09 PM PDT 1 hr 19 mins

./test2.sh (with -l 1) Sun 12 Sep 2021 09:47:55 PM PDT 581 Sun 12 Sep 2021 11:00:31 PM PDT 1 hr 13 mins

Memory/CPU usage seems to be the same using the limit feature, so shoving it into a dictionary, at least in limited testing so far doesn't seem to cause any issues.

I don't think the limit feature is making it any/much quicker, but it didn't add to the time which I was worried about. Will look into threads next.

xnih commented 3 years ago

Fixed a number of bugs in the code I implemented, so maybe threads next version. Notice more fingerprints for both runs:

./test1.sh Mon 13 Sep 2021 04:03:47 PM PDT 5009429 Mon 13 Sep 2021 05:33:02 PM PDT 1 hr 30 mins

./test2.sh Mon 13 Sep 2021 06:34:09 PM PDT 694 Mon 13 Sep 2021 08:05:00 PM PDT 1 hour 29 mins

xnih commented 3 years ago

running code in production for over a month with no issues, closing.