F-Stack / f-stack

F-Stack is an user space network development kit with high performance based on DPDK, FreeBSD TCP/IP stack and coroutine API.
http://www.f-stack.org
Other
3.87k stars 898 forks source link

The f-stack network port bridge performance test deteriorates when Iptables is not enabled on Linux ? #675

Closed jiaokailun closed 2 years ago

jiaokailun commented 2 years ago

Environment configuration

lo0: flags=8008<LOOPBACK,MULTICAST> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> groups: lo nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> f-stack-0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3<RXCSUM,TXCSUM> ether 0:22:46:33:c8:7a inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::222:46ff:fe33:c87a prefixlen 64 scopeid 0x2 nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> f-stack-1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=3<RXCSUM,TXCSUM> ether 0:22:46:36:9b:21 inet 192.168.1.4 netmask 0xffffff00 broadcast 192.168.1.255 inet6 fe80::222:46ff:fe36:9b21 prefixlen 64 scopeid 0x3 nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether 58:9c:fc:10:ff:a3 id 0:0:0:0:0:0 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 0:0:0:0:0:0 priority 32768 ifcost 0 port 0 member: f-stack-1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 3 priority 128 path cost 2000000 member: f-stack-0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 2 priority 128 path cost 2000000 groups: bridge nd6 options=9<PERFORMNUD,IFDISABLED>

test code 1 #include 2 #include 3 #include 4 #include 5 #include 6 #include <sys/types.h> 7 #include <sys/socket.h> 8 #include <arpa/inet.h> 9 #include 10 #include 11 12 #include "ff_config.h" 13 #include "ff_api.h" 14 15 #define MAX_EVENTS 512 16 int loop1(void arg){ 17 18 } 19 int main(int argc,char argv[]){ 20 ff_init(argc,argv); 21 ff_run(loop1,NULL); 22 }

test method

Bridge the two network ports on the server and use the packet sending tester to upload traffic .

test result

The 64KB packet throughput is 10.80%,The 256KB packet throughput is 34.85%,The 512KB packet throughput is 63.48%(not compile ipfw) frame size -----------------64 256 512 pair1 TxTput(fps) TxTput 10.80 34.85 63.48

In contrast, I use DPDK's skeleton test code,the test results are as follows。 frame size -----------------64 256 512 pair1 TxTput(fps) TxTput 89.36. 100.00 100.00

Linux kernel protocol stack, configure network port bridge disable iptables firewall, the result is as follows . frame size -----------------64 256 512 pair1 TxTput(fps) TxTput 35.29 91.39 100.00

issue

Is there any configuration that would make the F-Stack perform better? Is there a flaw in my experiment?

jfb8856606 commented 2 years ago

F-Stack has a complete protocol stack, can support L2(bridge) and L3(IP) forwoard.

But F-Stack optimized for L7 APP,will do a lot of extra operations while L2/L3 forward, so you shoudn't use F-Stack to do L2/L3 forward, it is recommended to use DPDK only or XDP, and Packet Filter Module can implement your own or find an open source implementation.