search

emmericp / MoonGen

MoonGen is a fully scriptable high-speed packet generator built on DPDK and LuaJIT. It can saturate a 10 Gbit/s connection with 64 byte packets on a single CPU core while executing user-provided Lua scripts for each packet. Multi-core support allows for even higher rates. It also features precise and accurate timestamping and rate control.
MIT License
1.04k stars 234 forks source link

l3-loadlatency.lua - Not detecting RX rate #257

Open 108anup opened 4 years ago

108anup commented 4 years ago

When I run load-latency flow on moongen-simple, I get both RX and TX measurements, while when I run examples/l3-loadlatency.lua I only get TX rate measurements and not RX rate measurements.

I observed that when I am doing moongen-simple I can't see the packets in tcpdump, while when I am running l3-loadlatency, I can see packets in tcpdump. Thus, the packets are going to the kernel rather than going to the packet generator. Now the dpdk filter that decides what to send to the kernel and what to send to the application (in case of bifurcated driver) is not exposed in the l3-loadlatency script. Could this be an issue in the way dpdk filters are setup internally in libmoon. Or am I doing some wrong configuration.

I am using Mellanox Connect X3 NIC (tried with both OFED 4.1 and 4.4, fw: 2.42) (Both at sender and DUT). At the DUT, I swap tcp ports, ip addresses and MAC addresses. I have hardcoded dst mac address in the moongen scripts.

Exactly the same routing config works with Intel NICs (XL710 for 40GbE QSFP+). Also, on the Intel NIC, I am not getting any timestamps. Generally intel NICs support H/W timestamps right?

Sorry, If I said something stupid (fairly new to this).

emmericp commented 4 years ago

Problems with DPDK setup are out of scope here; configure your mellanox NICs to not do any bifurcation.

Intel XL710 has some firmware versions with broken PTP support, try updating

108anup commented 4 years ago

Okay, Also in general, what does the RX expect the packet headers to be in terms of the IP, MAC, Ports etc. because in case of Intel and with moongen-simple if I just swap MAC addresses and don't swap IP, ports on the DUT things still work.

Might I ask, why exactly are dpdk setup issues out of scope?

I will check how things go without bifurcation, but in general this should work even with bifurcation right? (Most online resources are on the lines of the application setting the flow filters in flow isolation mode and not being a static setting for the NIC) In other words how do I globally turn off flow steering? If I use ethtool, will that be honored by the dpdk application? (doubt is based on this: https://dev.dpdk.narkive.com/ORFRhNVt/dpdk-dev-mellanox-flow-steering)

Thanks for the prompt response! (Is there a better way of contacting you than github issues?).

108anup commented 4 years ago

Also, why do things work with moongen-simple and not with the full API?