Absolute values for comparability

[aparcar]

Hi, this project looks fun, however I'm a bit unsure about the use-case. It gives a nice relative idea how how much faster/slower devices are within this specific test setup, however unlikely to represent real data since it does not incorporate NAT performance but adds the overhead of running two iPerf instances.

Since the benchmark shows that i.e. a network runs fine around 10Gbit/s, wouldn't it be a nice (additional) test-setup to test for real data, too? My idea would be to use network namespaces and two network cards (i.e. USB) on a strong device (i.e. Laptop) to route through a router and then compare the results. Since most laptops and USB ethernet adapters easily push 1Gbit/s and routers hardly ever, it would be rather simple to have valid results even with slightly different hardware.

Would that be of interested? I think the OpenWrt community could profit from something like that and even vendors should run their tests on a reproducible set.

[cyyself]

It gives a nice relative idea how how much faster/slower devices are within this specific test setup, however unlikely to represent real data since it does not incorporate NAT performance but adds the overhead of running two iPerf instances.

Yeah. I also mentioned this here: https://forum.openwrt.org/t/a-wireguard-comparison-db/187586/110?u=cyyself

My idea would be to use network namespaces and two network cards (i.e. USB) on a strong device (i.e. Laptop) to route through a router and then compare the results.

Yeah. But this requires a complex setup. We can't collect as many results as we do now. We can have a separate result board for this benchmark in a real environment if you can do it.

Also, there is a more straightforward way to use Cloudflare Warp, which provides public wireguard tunnels to reach the internet: we can override the reserved bytes on the wg packet to use kernel wireguard instead of Cloudflare's userspace client. As the largest CDN provider in the world, it might be capable of providing enough speed to reach your WAN ISP limit in most countries. The point is that I don't know if an ISP with a 1Gbps download speed subscription is common in the world.

[aparcar]

I don't know if an ISP with a 1Gbps download speed subscription is common in the world.

Living in Germany, the government considers 50Mbit/s as good enough...

I'll think about it and may come up with something which would be working locally. Thanks for your time.

[cyyself]

I don't know if an ISP with a 1Gbps download speed subscription is common in the world.

Living in Germany, the government considers 50Mbit/s as good enough...

Oh.... In China, 10G PON-based fiber connection is very common. About 23.0% of home broadband users have Download Speed >= 1000Mbps in 2023. However, we can reach Cloudflare at just about 100Mbps since they have no servers in China. The nearest servers might be Hong Kong or Japan, or even route the traffic to Los Angeles on some ISPs. The latency also ranges from 15-200ms, depending on different ISPs.

I'll think about it and may come up with something that would be working locally. Thanks for your time.

Another suggestion is assuming the server and the client on the router have no significant performance impact. Still, ethernet MAC hardware and driver will influence the performance; we can set up switch configuration on the router hardware and split two LAN ports to separate VLAN to bridge to different netns and then set up the topology like this:

+-----------+              +-----------+
|  netns 1  |              |  netns 2  |
|    LAN1-----(RJ45 Cable)------LAN2   |
|           |              |           |
| wireguard |              | wireguard |
| iperf3    |              | iperf3    |
+-----------+              +-----------+

In this case, a single RJ45 cable connected to 2 unused LAN ports would be enough. Since many routers supported by OpenWRT have a switch driver that supports DSA (Distributed Switch Architecture), writing a script to find unused LAN ports and setting up a VLAN like this might be easy.