search

phaag / nfdump

Netflow processing tools
Other
765 stars 201 forks source link

Separate NSEL data from usual NetFlow data #274

Closed antiuser closed 3 years ago

antiuser commented 3 years ago

Hi there! How can i separate NSEL (NEL) data from usual NetFlow data (with bytes rx/tx) with nfcapd? Im trying to run nfcapd with following parameters /usr/local/bin/nfcapd -w -z -T all,-1,-2,-9,-10,-11,-12,-14,-15,-20,-29,-30,-31,-32 -D -S 8 -l /mnt/flow/lisg1 -p 2056 -P /var/run/nfcapd-lisg1.pid -T nsel -l /mnt/nsel/lisg1 But data writing only to /mnt/nsel/lisg1, but nothing to /mnt/flow/lisg1 Our goal to minimize netflow data storage, earlier we used flow-tools (NETFLOW v5 without nsel) and have about 15 MBytes in 5 minutes, instead of nfcapd with 100+ MBytes (NETFLOW v9 with enabled NSEL) in 5 minutes. What can you recommend?

phaag commented 3 years ago

You can specify only one -l directory. You can split data based on export IP by using multiple -n <..> options. The data size is minimised automatically. Even if you specify -Tall nfcapd stores only those extensions in common with the exporter. Furthermore user compression -y (LZ4 compression). v9 sends way more information than v5, therefore you will get larger files. If you don't need this information configure your router for v5 or leave out any -T option, which pretty much reduces data to the v5 fields.

phaag commented 3 years ago

I close the ticked. If you have more questions, feel free to re-open it.