search

phaag / nfdump

Netflow processing tools
Other
771 stars 202 forks source link

nfdump server demands and optimizations #258

Closed thezoggy closed 3 years ago

thezoggy commented 3 years ago

Our nfsen/nfdump box is a 48 core box (Intel Xeon CPU E7-8857 v2 @ 3.00GHz) with 780G ram with 52T of ssd.

It has worked well for a few years now but over time with our network growth we see nfsen 20-30 mins lag behind realtime. We see all cores maxxed out and before we decide to try and just throw more hardware at the issue (move to vm) is there any tips/tricks that I can refer to about optimizing our setup?

Also, is there any way for nfdump/nfsen to benefit from having a multi-server approach (cluster) style design?

thezoggy commented 3 years ago

also, did nfsen dev ever get moved to github and is nfsen-1.3.8 still the latest ?

phaag commented 3 years ago

First of all, I am impressed that nfdump runs with that amount of load :) There are several issues here. First of all nfdump-1.6.x does not make use of threading. This may slow down processing of flows. nfdump-1.7.x is threaded, so it makes use of processing things in parallel. 1.7-beta is almost complete and testing can start soon. As of nfsen: The is considered legacy and there will be one last version - most likely 1.4, which will properly interact with nfdump-1.7. I'd like to shift away from nfsen based on perl and move towards Grafana as a frontend and maybe a golang backend, which performce better than perl - but this is still vapour ware. After all, I would be interested to get a sneak view into the reasons, why this lag exists, if possible. I am also discuss this outside GitHub, if you want to preserve your setup details.

thezoggy commented 3 years ago

Thats great news about 1.7, it probably would help everything a bit as thats probably a big bottleneck at times.

About reaching you outside of github, can I email you at your switch(.)ch address?

phaag commented 3 years ago

Look at the authors file :)

thezoggy commented 3 years ago

Hey there, sorry for the delay. I'll send you an email this week.