search

hermit-os / hermit-rs

Hermit for Rust.
Apache License 2.0
1.69k stars 86 forks source link

very low TCP performance #282

Open n-eiling opened 2 years ago

n-eiling commented 2 years ago

I would expect performance when reading/writing from/to TCP to be in the GB/s. I wrote a small program to test the bandwidth: https://git.rwth-aachen.de/niklas.eiling/rust-network-benchmark Running it without rusty-hermit (i.e., using a local loopback socket) yields approx. 5 GiB/s bandwidth. When running it in rusty-hermit I only get the following bandwidths: Write BW: 0.039 MiB/s Read BW: 0.53 MiB/s

My program is very similar to this benchmark in the rusty-hermit repo: https://github.com/hermitcore/rusty-hermit/tree/master/benches/netbench This also shows very bad performance.

stlankes commented 2 years ago

We seem to solve this issue. I run a few tests on our test machine. The server is running on the host (Rocky Linux). The client is running in a VM. I used a TAP device as interface between guest and host and build the benchmark as follow:

$ cargo build -Zbuild-std=core,alloc,std,panic_abort -Zbuild-std-features=compiler-builtins-mem --target x86_64-unknown-linux-gnu --bin server-bw --release
$ cargo build -Zbuild-std=core,alloc,std,panic_abort -Zbuild-std-features=compiler-builtins-mem --target x86_64-unknown-hermit --bin client-bw --release

The TAP device has been configured as described in the README. The server was started on the host as follows:

$ target/x86_64-unknown-linux-gnu/release/server-bw -k 1024 -r 10000 -b 0

The flag -k describe the packet size. The client was started in a VM as follows:

$ sudo /usr/libexec/qemu-kvm -smp 1 -cpu host -display none -m 1G -serial stdio -netdev tap,id=net0,ifname=tap10,script=no,downscript=no,vhost=off -device virtio-net-pci,netdev=net0,disable-legacy=on -kernel loader/target/x86_64/release/rusty-loader -initrd target/x86_64-unknown-hermit/release/client-bw -append '-- -a 10.0.5.1 -k 1024 -r 10000 -b 0'

I have varied the packet size and get the following bandwidths:

# number        RustyHermit
# of bytes      in MBit/s
1               19.3
2               31.4
4               61.5 
8               121.9
16              222.0
32              405.3
64              592.5
128             868.6
256             1279.4 
512             2754.1
1024            2844.5
2048            2797.9
4096            2661.2

The MTU is defined as 1500 Bytes. I measured the peek bandwidth at 1024 because a packet with a size of 2048 is spilt into 2 transfer units.

@n-eiling Can you confirm my results?