search

sflow / host-sflow

host-sflow agent
http://sflow.net
Other
146 stars 55 forks source link

Dynamic sampling rate for 10Gbps #26

Open nidotech opened 6 years ago

nidotech commented 6 years ago

I've a problem with very inaccurate sampling rate for 10Gbps link with low traffic (less than 500Mbps). Is it possible to implement a dynamic adjusted sampling rate into hsflowd?

sflow commented 6 years ago

That approach leads to statistical bias in the results. However the analysis is usually not very sensitive to sampling rate, so can you explain more about the problem? If your application really needs the finer granularity, then why not set a more aggressive sampling rate? The default for 10G would be 1-in-10000, but in /etc/hsflowd.conf you can set something like:

sampling.10G = 5000

nidotech commented 5 years ago

Since i'm measuring the traffic on a edge device, i've changed to the following rules in order to take care of sampling instead of pcap. However, it is still very inaccurate.

*raw -A PREROUTING -j NOTRACK -A OUTPUT -j NOTRACK

-A FORWARD -i enp101s0f0 -m statistic --mode random --probability 0.0025 -j NFLOG --nflog-group 1 --nflog-prefix SFLOW -A FORWARD -o enp101s0f0 -m statistic --mode random --probability 0.0025 -j NFLOG --nflog-group 1 --nflog-prefix SFLOW

/etc/hsflowd.conf polling = 20 nflog { group = 1 probability = 0.0025}

sflow commented 5 years ago

To run a calibration test, it works well to set up a constant flow with a moderate traffic level. For example, if there is a target IP you can ping, then something like this will generate 100 packets/sec in each direction:

sudo ping -i 0.01

With this you should see a sample from that flow every 4 seconds (on average) or around 15 samples/min. If you don't see that then there must be (1) a bottleneck where samples are dropped or (2) a problem with nflog or (3) a bug in hsflowd.

For a more comprehensive calibration-check I recommend the sflow-test app in sFlow-RT: https://blog.sflow.com/2015/11/sflow-test.html

Let me know how it goes.

nidotech commented 5 years ago

Does the tcp offload or other nic offload functions affecting the sampling collection?

sflow commented 5 years ago

Yes, I would expect TCP-offload to affect it. Are you seeing 64KB packet-samples, or do the samples disappear altogether?

If you suspect that hsflowd is rejecting the samples somehow, then running with debug logging might shed light:

sudo systemctl stop hsflowd sudo hsflowd -dddd 2>&1

The above will print lines of output for every sample that hsflowd receives.