rootless-containers / bypass4netns

[Experimental] Accelerates slirp4netns using SECCOMP_IOCTL_NOTIF_ADDFD. As fast as `--net=host`.
https://medium.com/nttlabs/accelerating-rootless-container-network-29d0e908dda4
Apache License 2.0
131 stars 6 forks source link

Rewrite in Golang and PoC implementation of host -> netns port-forwading #9

Closed naoki9911 closed 2 years ago

naoki9911 commented 2 years ago

@AkihiroSuda Thank you for your comments! I improved the code as you mentioned.

AkihiroSuda commented 2 years ago

Benchmark data from https://github.com/rootless-containers/bypass4netns/runs/5063766110?check_suite_focus=true (Vagrant on GHA)

    default: ===== Benchmark: netns -> host With bypass4netns =====
    default: + systemd-run --user --unit run-iperf3 iperf3 -s
    default: Running as unit: run-iperf3.service
    default: + nerdctl run --security-opt seccomp=/tmp/seccomp.json -d --name test public.ecr.aws/docker/library/alpine:3.15 sleep infinity
    default: 880bd71a92939c728f36872cdaa845139d8b0f31aaea2d9cadef4462d98af15c
    default: + nerdctl exec test apk add --no-cache iperf3
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
    default: (1/1) Installing iperf3 (3.10.1-r0)
    default: Executing busybox-1.34.1-r3.trigger
    default: OK: 6 MiB in 15 packages
    default: ++ cat /tmp/host_ip
    default: + nerdctl exec test iperf3 -c 192.168.6.15
    default: Connecting to host 192.168.6.15, port 5201
    default: [  5] local 192.168.6.15 port 44968 connected to 192.168.6.15 port 5201
    default: [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    default: [  5]   0.00-1.00   sec  3.02 GBytes  26.0 Gbits/sec    3   2.00 MBytes
    default: [  5]   1.00-2.00   sec  2.77 GBytes  23.8 Gbits/sec    0   2.31 MBytes
    default: [  5]   2.00-3.00   sec  3.15 GBytes  27.0 Gbits/sec    1   2.44 MBytes
    default: [  5]   3.00-4.00   sec  3.04 GBytes  26.1 Gbits/sec    3   2.44 MBytes
    default: [  5]   4.00-5.00   sec  2.93 GBytes  25.2 Gbits/sec    1   2.44 MBytes
    default: [  5]   5.00-6.00   sec  2.96 GBytes  25.4 Gbits/sec    0   2.44 MBytes
    default: [  5]   6.00-7.00   sec  1.77 GBytes  15.2 Gbits/sec    0   2.94 MBytes
    default: [  5]   7.00-8.00   sec  2.81 GBytes  24.2 Gbits/sec    2   3.06 MBytes
    default: [  5]   8.00-9.00   sec  2.69 GBytes  23.1 Gbits/sec    0   3.06 MBytes
    default: [  5]   9.00-10.00  sec  3.00 GBytes  25.8 Gbits/sec    1   3.06 MBytes
    default: - - - - - - - - - - - - - - - - - - - - - - - - -
    default: [ ID] Interval           Transfer     Bitrate         Retr
    default: [  5]   0.00-10.00  sec  28.1 GBytes  24.2 Gbits/sec   11             sender
    default: [  5]   0.00-10.00  sec  28.1 GBytes  24.2 Gbits/sec                  receiver
    default: 
    default: iperf Done.
    default: + nerdctl rm -f test
    default: test
    default: ===== Benchmark: netns -> host Without bypass4netns (for comparison) =====
    default: + nerdctl run -d --name test public.ecr.aws/docker/library/alpine:3.15 sleep infinity
    default: 9a10e8992f7b1d91df338573553f0d7b0c465330911ac19a19edbe5b70563725
    default: + nerdctl exec test apk add --no-cache iperf3
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
    default: (1/1) Installing iperf3 (3.10.1-r0)
    default: Executing busybox-1.34.1-r3.trigger
    default: OK: 6 MiB in 15 packages
    default: ++ cat /tmp/host_ip
    default: + nerdctl exec test iperf3 -c 192.168.6.15
    default: Connecting to host 192.168.6.15, port 5201
    default: [  5] local 10.4.0.3 port 33244 connected to 192.168.6.15 port 5201
    default: [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    default: [  5]   0.00-1.00   sec  40.0 MBytes   335 Mbits/sec    0   81.3 KBytes
    default: [  5]   1.00-2.00   sec  40.0 MBytes   336 Mbits/sec    0   81.3 KBytes
    default: [  5]   2.00-3.00   sec  42.5 MBytes   357 Mbits/sec    0   81.3 KBytes
    default: [  5]   3.00-4.00   sec  42.5 MBytes   357 Mbits/sec    0   81.3 KBytes
    default: [  5]   4.00-5.00   sec  38.8 MBytes   325 Mbits/sec    0   81.3 KBytes
    default: [  5]   5.00-6.00   sec  25.0 MBytes   210 Mbits/sec    0   81.3 KBytes
    default: [  5]   6.00-7.00   sec  28.8 MBytes   241 Mbits/sec    0   81.3 KBytes
    default: [  5]   7.00-8.00   sec  35.0 MBytes   294 Mbits/sec    0   81.3 KBytes
    default: [  5]   8.00-9.00   sec  37.5 MBytes   315 Mbits/sec    0   81.3 KBytes
    default: [  5]   9.00-10.00  sec  40.0 MBytes   335 Mbits/sec    0   81.3 KBytes
    default: - - - - - - - - - - - - - - - - - - - - - - - - -
    default: [ ID] Interval           Transfer     Bitrate         Retr
    default: [  5]   0.00-10.00  sec   370 MBytes   310 Mbits/sec    0             sender
    default: [  5]   0.00-10.00  sec   367 MBytes   307 Mbits/sec                  receiver
    default: 
    default: iperf Done.
    default: + nerdctl rm -f test
    default: test
    default: ===== Benchmark: host -> netns With bypass4netns =====
    default: + nerdctl run --security-opt seccomp=/tmp/seccomp.json -d --name test public.ecr.aws/docker/library/alpine:3.15 sleep infinity
    default: 929de673a2d15d6c4ea7ea4766f1ef505983a6898a7539a8bb98c00b764effdf
    default: + nerdctl exec test apk add --no-cache iperf3
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
    default: (1/1) Installing iperf3 (3.10.1-r0)
    default: Executing busybox-1.34.1-r3.trigger
    default: OK: 6 MiB in 15 packages
    default: + systemd-run --user --unit run-iperf3-netns nerdctl exec test iperf3 -s -4
    default: Running as unit: run-iperf3-netns.service
    default: + sleep 1
    default: ++ cat /tmp/host_ip
    default: + iperf3 -c 192.168.6.15 -p 8080
    default: Connecting to host 192.168.6.15, port 8080
    default: [  5] local 192.168.6.15 port 37288 connected to 192.168.6.15 port 8080
    default: [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    default: [  5]   0.00-1.00   sec  2.74 GBytes  23.5 Gbits/sec    0   3.06 MBytes
    default: [  5]   1.00-2.00   sec  2.93 GBytes  25.2 Gbits/sec    1   3.25 MBytes
    default: [  5]   2.00-3.00   sec  3.12 GBytes  26.8 Gbits/sec    2   3.25 MBytes
    default: [  5]   3.00-4.00   sec  3.33 GBytes  28.6 Gbits/sec    0   3.25 MBytes
    default: [  5]   4.00-5.00   sec  1.89 GBytes  16.3 Gbits/sec    1   3.25 MBytes
    default: [  5]   5.00-6.00   sec  2.95 GBytes  25.4 Gbits/sec    0   3.25 MBytes
    default: [  5]   6.00-7.00   sec  3.14 GBytes  26.9 Gbits/sec    0   3.25 MBytes
    default: [  5]   7.00-8.00   sec  3.04 GBytes  26.1 Gbits/sec    3   3.25 MBytes
    default: [  5]   8.00-9.00   sec  3.11 GBytes  26.7 Gbits/sec    0   3.25 MBytes
    default: [  5]   9.00-10.00  sec  3.06 GBytes  26.3 Gbits/sec    0   3.25 MBytes
    default: - - - - - - - - - - - - - - - - - - - - - - - - -
    default: [ ID] Interval           Transfer     Bitrate         Retr
    default: [  5]   0.00-10.00  sec  29.3 GBytes  25.2 Gbits/sec    7             sender
    default: [  5]   0.00-10.00  sec  29.3 GBytes  25.2 Gbits/sec                  receiver
    default: 
    default: iperf Done.
    default: + nerdctl rm -f test
    default: test
    default: ===== Benchmark: host -> netns Without bypass4netns (for comparison) =====
    default: + nerdctl run -d --name test -p 8080:5201 public.ecr.aws/docker/library/alpine:3.15 sleep infinity
    default: 062fb296f9970eb15c934d085934a78aa0957b1ed48ca8e396232c1df23865cb
    default: + nerdctl exec test apk add --no-cache iperf3
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/main/x86_64/APKINDEX.tar.gz
    default: fetch https://dl-cdn.alpinelinux.org/alpine/v3.15/community/x86_64/APKINDEX.tar.gz
    default: (1/1) Installing iperf3 (3.10.1-r0)
    default: Executing busybox-1.34.1-r3.trigger
    default: OK: 6 MiB in 15 packages
    default: + systemd-run --user --unit run-iperf3-netns2 nerdctl exec test iperf3 -s -4
    default: Running as unit: run-iperf3-netns2.service
    default: + sleep 1
    default: ++ cat /tmp/host_ip
    default: + iperf3 -c 192.168.6.15 -p 8080
    default: Connecting to host 192.168.6.15, port 8080
    default: [  5] local 192.168.6.15 port 37292 connected to 192.168.6.15 port 8080
    default: [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
    default: [  5]   0.00-1.00   sec  1.57 GBytes  13.4 Gbits/sec    0   3.18 MBytes
    default: [  5]   1.00-2.00   sec  1.38 GBytes  11.9 Gbits/sec    0   3.18 MBytes
    default: [  5]   2.00-3.00   sec  1016 MBytes  8.49 Gbits/sec    0   3.18 MBytes
    default: [  5]   3.00-4.00   sec  1.39 GBytes  12.0 Gbits/sec    0   3.18 MBytes
    default: [  5]   4.00-5.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.18 MBytes
    default: [  5]   5.00-6.00   sec  1.22 GBytes  10.5 Gbits/sec    0   3.18 MBytes
    default: [  5]   6.00-7.00   sec  1.16 GBytes  9.95 Gbits/sec    0   3.18 MBytes
    default: [  5]   7.00-8.00   sec  1.04 GBytes  8.97 Gbits/sec    0   3.18 MBytes
    default: [  5]   8.00-9.00   sec  1.15 GBytes  9.83 Gbits/sec    0   3.18 MBytes
    default: [  5]   9.00-10.00  sec  1.08 GBytes  9.24 Gbits/sec    0   3.18 MBytes
    default: - - - - - - - - - - - - - - - - - - - - - - - - -
    default: [ ID] Interval           Transfer     Bitrate         Retr
    default: [  5]   0.00-10.00  sec  12.1 GBytes  10.4 Gbits/sec    0             sender
    default: [  5]   0.00-10.00  sec  12.1 GBytes  10.4 Gbits/sec                  receiver
    default: 
    default: iperf Done.
    default: + nerdctl rm -f test
    default: test