search

stamparm / maltrail

Malicious traffic detection system
MIT License
6.44k stars 1.07k forks source link

Regularly "killed" #121

Closed gregfr closed 7 years ago

gregfr commented 7 years ago

Greetings I've installed sensor & server on the same system. Every few days, the sensor stops like this:

[o] running... (a few days pass)

[i] updating trails (this might take a while)...
 [o] 'https://reputation.alienvault.com/reputation.generic'
 [o] 'http://cybercrime-tracker.net/ccam.php'
 [o] 'https://www.badips.com/get/list/any/2?age=7d'
 [o] 'http://osint.bambenekconsulting.com/feeds/c2-dommasterlist-high.txt'
 [o] 'http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist-high.txt'
 [o] 'http://osint.bambenekconsulting.com/feeds/dga-feed.txt'
Killedogress: 5/63 (7%)

What am I doing wrong? Thanks in advance

stamparm commented 7 years ago

That DGA feed is sometimes too large for a low memory system. I bet you have a low memory (e.g. 256 MB) on that sensor machine, right?

On Feb 7, 2017 08:45, "gregfr" notifications@github.com wrote:

Greetings I've installed sensor & server on the same system. Every few days, the sensor stops like this:

[o] running... (a few days pass)

[i] updating trails (this might take a while)... [o] 'https://reputation.alienvault.com/reputation.generic' [o] 'http://cybercrime-tracker.net/ccam.php' [o] 'https://www.badips.com/get/list/any/2?age=7d' [o] 'http://osint.bambenekconsulting.com/feeds/c2-dommasterlist-high.txt' [o] 'http://osint.bambenekconsulting.com/feeds/c2-ipmasterlist-high.txt' [o] 'http://osint.bambenekconsulting.com/feeds/dga-feed.txt' Killedogress: 5/63 (7%)

What am I doing wrong? Thanks in advance

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/stamparm/maltrail/issues/121, or mute the thread https://github.com/notifications/unsubscribe-auth/AA4P03JfCr15dMqT_t7GRc9Vp1DKWKw7ks5raCEcgaJpZM4L5KcN .

gregfr commented 7 years ago

Thanks for you answer. It's a 512M VM, I'll try to monitor the RAM at the point where the sensor gets killed.

stamparm commented 7 years ago

With the latest revision FAR less memory is used on this feed. It is downloaded in gzip format (cca. 6MB) and read/decompressed/parsed line by line. Previously the whole uncompressed file has been parsed inside the memory (cca. 111MB).

gregfr commented 7 years ago

Thanks that is nice :)