search

pavel-odintsov / fastnetmon

FastNetMon - very fast DDoS sensor with sFlow/Netflow/IPFIX/SPAN support
https://fastnetmon.com
GNU General Public License v2.0
3.43k stars 568 forks source link

Detect e-mail spamming #278

Closed henry-spanka closed 7 years ago

henry-spanka commented 9 years ago

It would be cool to detect outgoing e-mail spamming and trigger block script with specific reason like smtp_spam.

pavel-odintsov commented 9 years ago

Nice suggestion! But first of all I need fix flow tracker it's very slow and eat whole CPU on very slow speed :(

Thus, I have finished some tests in src/tests/traffic_structures_performance_tests.cpp:

./a.out 
std::map: 0.8 mega ops per second
tbb::concurrent_unordered_map: 0.9 mega ops per second
boost::unordered_map: 1.1 mega ops per second
std::unordered_map C++11: 2.0 mega ops per second
std::unordered_map C++11 preallocated: 3.7 mega ops per second
std::vector preallocated: 49.6 mega ops per second

And I need new very fast structure which could achieve ~14 mpps for single CPU core...... Maybe you can help with it?

bolt1777 commented 9 years ago

Dear Pavel,

What exact OS in question? Does the issue persists on FreeBSD with netmap? I see freebsd+netmap are working fine. I got up to 1Gbit traffic and detection is working well.

pavel-odintsov commented 9 years ago

This issue will arrive if you have /8 network in monitoring list. Thats why it's enough rare and everything should work perfectly for smaller networks :)

bolt1777 commented 9 years ago

I got /19

bolt1777 commented 9 years ago

I'm getting many errors like this:

2015-06-15 14:20:49,017 [INFO] Time from last run of speed_recalc is soooo big, we got ugly lags: 2 2015-06-15 14:21:16,000 [INFO] Time from last run of speed_recalc is soooo big, we got ugly lags: 2 2015-06-15 14:22:41,005 [INFO] Time from last run of speed_recalc is soooo big, we got ugly lags: 2 Does it reflect to fastnetmon performance somehow?

pavel-odintsov commented 9 years ago

Nop, it's not a critical error, only soft warning about some slowdown. Could be related with other services running on same box.

henry-spanka commented 9 years ago

I would say it isn't really an issue. Who is monitoring more than a /8 with fastnetmon? In like 99.99% use cases not needed. And IF needed, you can split fastnetmon across multiple nodes and split the /8 subnet. Doesn't really make sense to use a /8 CIDR because you have only one broadcast domain. If i'm wrong, correct me ;)

pavel-odintsov commented 9 years ago

Really, I have few customers with /10 and so often users want use private /8 networks used for NAT'ed customers :) FastNetMon has really ugly design here and I need to fix it!

henry-spanka commented 9 years ago

That's true. The code/structure is not fast enough to do something like deep packet inspection at wire speed(10GE).

pavel-odintsov commented 9 years ago

Well, since this commit https://github.com/FastVPSEestiOu/fastnetmon/commit/eae33ce39f2128b68c3d2e8340ca2e197c12126d we have got significant speed up :)

Phinitris, could you add me in skype: energy_true?

henry-spanka commented 9 years ago

Done ;)

pavel-odintsov commented 9 years ago

It's also possible with modern nDPI feature :)

pavel-odintsov commented 7 years ago

This feature could be implemented only with port mirror option. Unfortunately, the only very small number of FNM users use it. And this task lies outside DDoS mitigation World and I'm going to drop this request from work-queue