search

aabc / ipt-netflow

Netflow iptables module for Linux kernel (official)
https://github.com/aabc/ipt-netflow
499 stars 127 forks source link

ipt_netflow benchmarking #59

Open morphyno opened 7 years ago

morphyno commented 7 years ago

I have been testing ipt_netflow in promisc mode using tcpreplay (playing a pcap of 2000 iperf flows). Playing at various speeds (all less than the active timeout), I am noticing significant higher amount of packets and byte count from the collector listening to ipt_netflow.

When net.netflow.flush=1 is asserted, does it force ipt_netflow to purge all entries AND export them?

aabc commented 7 years ago

Yes.

morphyno commented 7 years ago

Is there a way to force the purge (to collector) without flushing the internal tables?

morphyno commented 7 years ago

This is sample output of what I'm running.

I'm playing 1000 packets from pcap using tcpreplay

Actual: 1000 packets (577912 bytes) sent in 14.09 seconds. Rated: 38500.0 Bps, 0.308 Mbps, 66.70 pps Flows: 92 flows, 6.13 fps, 992 flow packets, 8 non-flow Statistics for network device: p3p1 Successful packets: 1000 Failed packets: 0 Truncated packets: 0 Retried packets (ENOBUFS): 0 Retried packets (EAGAIN): 0

Below is the output of nfdump (collected from ipt_netflow)

Summary: total flows: 92, total bytes: 1690824, total packets: 2976, avg bps: 2554114, avg pps: 561, avg bpp: 568 Time window: 2016-08-23 15:05:00 - 2016-08-23 15:13:18 Total flows processed: 92, Blocks skipped: 0, Bytes read: 5392 Sys: 0.004s flows/second: 23000.0 Wall: 0.000s flows/second: 99567.1

The flows line up (92 bytes), but the bytes and total packets are way off. I have no fragmented packets anywhere either.

Below is my kernel settings

net.netflow.active_timeout = 1800 net.netflow.debug = 0 net.netflow.destination = 127.0.0.1:2055 net.netflow.flush = 0 net.netflow.hashsize = 655360 net.netflow.inactive_timeout = 15 net.netflow.maxflows = 2000000 net.netflow.promisc = 1 net.netflow.protocol = 5 net.netflow.refresh-rate = 20 net.netflow.scan-min = 1 net.netflow.sndbuf = 212992 net.netflow.timeout-rate = 30

aabc commented 7 years ago

Is there a way to force the purge (to collector) without flushing the internal tables?

What this mean?

morphyno commented 7 years ago

Is there another way to force ipt_netflow export to the collector without using net.netflow.flush?

aabc commented 7 years ago

It's always exporting.

morphyno commented 7 years ago

I'm just baffled where the extra bytes and packets coming are from

aabc commented 7 years ago

What extra bytes? total packets: 2976 != 1000 packets ? Check counters in iptables -L -n -v -x and stat in /proc/net/stat/ipt_netflow.

aabc commented 7 years ago

Also, if you replay on lo all packets are duplicated, but, you say network device: p3p1 which I don't know what it is.

morphyno commented 7 years ago

I'm playing out on p3p1, which is directly connected to p3p2 (in promisc mode) p3p2 is then forward to the NETFLOW module, which is exporting to 127.0.0.1:2055 nfcapd is listening on port 2055.

morphyno commented 7 years ago

This might be a collector issue, even when I moved to a remote collector, i'm seeing a tremendous amount of extra packets and bytes