Not much difference in speed

[Nuclear6]

Why is there not much difference when I test on a 32-core machine?

[atoonk]

Hi @Nuclear6, since you're using only one stream, you will only ever use one CPU core. That's how the Linux kernel works. It load balances network flows (5 tuples) over all available nic queues. So if you have only one flow, you'll use only one Queue and one CPU.

To ensure you're benefiting from multiple CPU cores, you need more flows by increasing the --streams flag. Also, read this section in readme to make sure you know how many network queues your NIC has and increase it.

test with ethtool -l enp4s0 to see what the max number of configurable tx queues are on your nic. Then set it to that number like for example ethtool -L enp4s0 tx 16. But make sure to check the max number supported. Also sometimes instead of tx, you need to use combined

let me know how it goes!

[Nuclear6]

Running this command ethtool -l enp4s0 gives unsupported output, why?

Channel parameters for enp4s0: Cannot get device channel parameters : Operation not supported

[atoonk]

Hmm, i guess that indicates your nic doesn't support multi-queue.. Also see this post: https://serverfault.com/questions/772380/how-to-tell-if-nic-has-multiqueue-enabled maybe also check cat /proc/interrupts | grep enp4s0 What nic do you have, and what driver?

In the meantime, try it with a veth interface; try this script. Then check the ethtool again. Likely that you can set multiple queues on that. This is what I used for most testing. then just use sudo ./go-pktgen --dstip 192.168.64.2 --method benchmark --duration 5 --payloadsize 64 --iface veth0 --streams 2

[Nuclear6]

use ethtool print nic info, Does this help?

driver: r8169 version: 5.15.0-100-generic firmware-version: rtl8125b-2_0.0.2 07/13/20 expansion-rom-version: bus-info: 0000:04:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no

[atoonk]

hard to say, you would need to look up the exact specs for your hardware version. Look for supported number of Queues or RSS (receivers side scaling) It's all hardware specific (maybe firmware?). Not something i can easily help with unfortunately.

ie. to go faster you need multiple streams --streams 4. To really benefit from multiple streams you need multiple queues on your nic.

did you to try the veth options i mentioned with the script? That will allow you to test it, see the difference with number of queues, etc.

[atoonk]

btw another way to check the available queues on your nic is mentioned in this issue

"eth2 has 8 tx queues and 8 rx queues"

ls /sys/class/net/eth2/queues/
rx-0  rx-1  rx-2  rx-3  rx-4  rx-5  rx-6  rx-7  tx-0  tx-1  tx-2  tx-3  tx-4  tx-5  tx-6  tx-7

[Nuclear6]

ls /sys/class/net/enp4s0/queues/
rx-0  tx-0

I haven't try the veth options, I'm afraid of damaging the network.

[atoonk]

ok, this shows indeed just one RX and one TX queue. a vethj interface should be safe: that script I mentioned creates a network namespace (a virtual network) on your machine. and then two veth interfaces, which are virtual network interfaces between the namespace and your main network stack.

ie. it's all local to the machine, and shouldn't leave the machine. Just double-check the IP addresses.

[Nuclear6]

Well, if that's the case, then your results are very convincing. I saw this project was included in the Go Weekly Report, congratulations to you.