pcapplusplus dpdk device performance is poor

seladb / PcapPlusPlus

PcapPlusPlus is a multiplatform C++ library for capturing, parsing and crafting of network packets. It is designed to be efficient, powerful and easy to use. It provides C++ wrappers for the most popular packet processing engines such as libpcap, Npcap, WinPcap, DPDK, AF_XDP and PF_RING.

https://pcapplusplus.github.io/

The Unlicense

2.63k stars 640 forks source link

pcapplusplus dpdk device performance is poor #1444

Closed nullbyte777 closed 3 weeks ago

nullbyte777 commented 4 weeks ago

how can I make my pcap++ dpdk application performance better ? currently when I only use regular device to capture packets performs better than dpdk device and I know it's not supposed to be that way.

I'm using vmware and ubuntu 20.04 and latest pcap++ version and dpdk version 21.11

in vmware I'm using two adapters which adapter1 is nat and the other one would be a dpdk port.

I only struggled with hugepages a bit even cleared hugepages and made a 1G with 3 pages which is in total 3G hugepages to see what happens and it only made it worse. packets weren't even comming in. I don't know what else I can do.

I'm using tcp reassebmly using dpdk device. without dpdk(regulare device) it works ok meaning for example if I download a file 10 times tcp reassembly without dpdk would receive 8 of them but with dpdk that would decrease to 2 or 4 of them.

I'm using vfio-pci as driver and I haven't touched not much of dpdk configs but only hugepages

seladb commented 3 weeks ago

@nullbyte777 I haven't tested PcapPlusPlus on VMWare with vfio-pci so I'm not sure which performance to expect. I have a few questions though:

When you say you tested your app with a "regular device", what does it mean? Did you test the same code with PcapLiveDevice or something else?
Did you test DPDK without PcapPlusPlus to capture packets? Did you see better performance?
Did you try DPDK with PcapPlusplus but without TCP reassembly? Did you see better performance?

nullbyte777 commented 3 weeks ago

@seladb

By regular device I mean PcapLiveDevice which I think by default it uses libpcap engine. and yes I tested the same code with PcapLiveDevice which performed ok. and I tested with dpdk which did not perform ok at all.
I don't know how to test dpdk without PcapPlusPlus to capture packets and see the packet drop and compare PcapLiveDevice and dpdk performance. if you could help me, that would be great.
I don't know how to do that since my metric is to pipe down packets from the device(I mean PcapLiveDevice or DpdkDevice) to tcpReassembly and there I can see if for example the whole file or the whole bytes are received correctly or not.

seladb commented 3 weeks ago

I don't know how to test dpdk without PcapPlusPlus to capture packets and see the packet drop and compare PcapLiveDevice and dpdk performance. if you could help me, that would be great.

I think you can run the sample applications provided with DPDK such as l2fwd or l3fwd and measure the throughput. You can read about these apps in DPDK documentation: https://doc.dpdk.org/guides/sample_app_ug/intro.html

I don't know how to do that since my metric is to pipe down packets from the device(I mean PcapLiveDevice or DpdkDevice) to tcpReassembly and there I can see if for example the whole file or the whole bytes are received correctly or not.

Did you try to run the DpdkExample-FilterTraffic example app and measure its performance?

nullbyte777 commented 3 weeks ago

@seladb, thank you so much for dedicating your time and sorry for the delayed response. I tested DpdkExample-FilterTraffic after you mentioned it and its performance was much better than my script, so I decided to use all of that standard in that script into my code and after doing that my performance got a lot better!

again thank you, it's a cool framework and I hope it gets better and better everyday.

seladb commented 3 weeks ago

@nullbyte777 thank you for the kind words and I'm glad you see a better performance. Let us know if you have more questions