search

luigirizzo / netmap

Automatically exported from code.google.com/p/netmap
BSD 2-Clause "Simplified" License
1.83k stars 533 forks source link

[question] Contention between several netmap bridges #448

Open Linerd opened 6 years ago

Linerd commented 6 years ago

Hi,

I was trying to measure the performance of netmap bridge with netmap-patched veth pairs (since VALE ports are non-blocking).

The topology setting is pkt-gen --- netmap-bridge --- pkt-gen (where --- indicates a patched veth pair).

I was able to achieve more than 100Gbps with 1500B packets with this setting. The netmap-bridge drains one CPU core.

However, when I instantiated another exact same flow, the CPU usage of each netmap-bridge instance is still at 100%, and the throughput for each flow is around only 30Gbps, combining to 60Gbps. With 2X cpu usage, I got 40% less aggregate throughput.

I wonder if anyone has the knowledge of why this is happening, and how such parallel setting can be improved. Ideally I would like to see the 100Gbps be distributed among different flows, even at the expense of higher overall CPU usages.

giuseppelettieri commented 6 years ago

You may be trashing some cache. You should experiment with pinning (taskset) the three processes in different combinations.

Linerd commented 6 years ago

@giuseppelettieri I placed the bridge instances into containers and was able to observe the actual throughput (4+Mpps, 50+Gbps, 1500B pkt) for a single core. However, when another such flow is created, both containers consume 100% CPU while delivering only 10+Gbps.