May I ask some question about the performance?

AlanYiNew commented 6 years ago

Currently I am using iperf to test a configuration that a router is directly connected with 3 machines. When I test using 1 machine sending packets to one of the other two, I could only reach a 1 Mbps while link is 1000 Mbps. It's running in userlevel. Is it a normal result? If I use iperf with 100 connection it can reach 95-105 Mbps.

bcronje commented 6 years ago

You should get much much better performance than that, so I suspect something else is amiss.

AlanYiNew commented 6 years ago

Interesting thing is that UDP can reach 870+ Mbps. Why would the TCP has such bad result with udp kind of like working? Is it possible to have a wrong configuration to let that happen like this? Can I post the configuration here?

bcronje commented 6 years ago

I suspect it might have something to do with offloading. On the router, disable offloading on all the Click router NICs. Run the following for all NICs that Click uses in the router:

ethtool --offload eth0 rx off              
ethtool --offload eth0 tx off 
ethtool --offload eth0 sg off
ethtool --offload eth0 tso off   
ethtool --offload eth0 gso off   
ethtool --offload eth0 gro off   
ethtool --offload eth0 lro off   
ethtool --offload eth0 rxvlan off
ethtool --offload eth0 txvlan off
ethtool --offload eth0 rxhash off

If you are just using standard Click without netmap or DPDK then make sure you are using PCAP instead of Linux method on FromDevice, PCAP gives better performance. So make sure your FromDevice elements look something like:

fd0::FromDevice(eth0, SNIFFER false, PROMISC true, METHOD PCAP);

The SNIFFER false is optional, this prevent Linux itself from passing the packet upstream to the Linux protocol stacks.

Hope that helps.

AlanYiNew commented 6 years ago

Really thanks for your help. Turning off offloading does help.

kohler / click

May I ask some question about the performance? #375