search

angt / glorytun

Multipath UDP tunnel
BSD 2-Clause "Simplified" License
683 stars 105 forks source link

Question about measurement #49

Closed user747 closed 5 years ago

user747 commented 5 years ago

Hi Angt great work thank you for this.

Happy with results I'm getting on 6 WAN (ADSL over PPPoE/LLC/SNAP 1446 MTU)

I'm wondering if I can squeeze out any more juice? Results look good, but wanted your opinion to see if they were in line with what you would expect.

iperf3 tells me each WAN can do around 5 Mbits/sec download, and 600Kbits/sec upload.

In a perfect world without overhead and clear weather, something like 30Mbit/sec down, and 3600Kbit/sec up.

Aggregated Download

ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 2.92 MBytes 24.5 Mbits/sec
[ 5] 1.00-2.00 sec 3.12 MBytes 26.2 Mbits/sec
[ 5] 2.00-3.00 sec 3.29 MBytes 27.6 Mbits/sec
[ 5] 3.00-4.00 sec 3.32 MBytes 27.9 Mbits/sec
[ 5] 4.00-5.00 sec 3.41 MBytes 28.6 Mbits/sec
[ 5] 5.00-6.00 sec 3.52 MBytes 29.5 Mbits/sec
[ 5] 6.00-7.00 sec 2.89 MBytes 24.3 Mbits/sec
[ 5] 7.00-8.00 sec 3.34 MBytes 28.0 Mbits/sec
[ 5] 8.00-9.00 sec 3.38 MBytes 28.4 Mbits/sec
[ 5] 9.00-10.00 sec 3.21 MBytes 26.9 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 33.4 MBytes 28.0 Mbits/sec 358 sender [ 5] 0.00-10.00 sec 32.4 MBytes 27.2 Mbits/sec receiver

Aggregated Upload

[ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 714 KBytes 5.85 Mbits/sec 85 21.0 KBytes
[ 5] 1.00-2.00 sec 189 KBytes 1.55 Mbits/sec 31 23.6 KBytes
[ 5] 2.00-3.00 sec 378 KBytes 3.10 Mbits/sec 15 22.3 KBytes
[ 5] 3.00-4.00 sec 189 KBytes 1.55 Mbits/sec 16 23.6 KBytes
[ 5] 4.00-5.00 sec 378 KBytes 3.10 Mbits/sec 16 26.2 KBytes
[ 5] 5.00-6.00 sec 378 KBytes 3.10 Mbits/sec 44 26.2 KBytes
[ 5] 6.00-7.00 sec 189 KBytes 1.55 Mbits/sec 62 18.4 KBytes
[ 5] 7.00-8.00 sec 189 KBytes 1.55 Mbits/sec 1 23.6 KBytes
[ 5] 8.00-9.00 sec 378 KBytes 3.10 Mbits/sec 38 22.3 KBytes
[ 5] 9.00-10.00 sec 189 KBytes 1.55 Mbits/sec 23 5.25 KBytes

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 3.10 MBytes 2.60 Mbits/sec 331 sender [ 5] 0.00-10.00 sec 2.71 MBytes 2.27 Mbits/sec receiver

    WAN1

Glorytun:

path UP status: OK

mtu: 1418 bytes rtt: 31.426 ms rttvar: 0.568 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1163489 packets total rx: 991371 packets

iPerf Download:

[ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 634 KBytes 5.20 Mbits/sec
[ 5] 1.00-2.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 2.00-3.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 3.00-4.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 4.00-5.00 sec 615 KBytes 5.04 Mbits/sec
[ 5] 5.00-6.00 sec 615 KBytes 5.04 Mbits/sec
[ 5] 6.00-7.00 sec 565 KBytes 4.63 Mbits/sec
[ 5] 7.00-8.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 8.00-9.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 9.00-10.00 sec 617 KBytes 5.05 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.25 MBytes 5.24 Mbits/sec 13 sender [ 5] 0.00-10.00 sec 6.00 MBytes 5.03 Mbits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 1.00-2.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 2.00-3.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 3.00-4.00 sec 83.0 KBytes 680 Kbits/sec
[ 5] 4.00-5.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 5.00-6.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 6.00-7.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 7.00-8.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 8.00-9.00 sec 74.9 KBytes 613 Kbits/sec
[ 5] 9.00-10.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 10.00-10.11 sec 8.17 KBytes 590 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.11 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.11 sec 785 KBytes 636 Kbits/sec receiver

    WAN2

path UP status: OK

mtu: 1418 bytes rtt: 32.856 ms rttvar: 0.501 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1167588 packets total rx: 996366 packets

iPerf Download:

[ ID] Interval Transfer Bandwidth Retr Cwnd [ 5] 0.00-1.00 sec 856 KBytes 7.01 Mbits/sec 0 93.9 KBytes
[ 5] 1.00-2.00 sec 626 KBytes 5.13 Mbits/sec 9 80.3 KBytes
[ 5] 2.00-3.00 sec 626 KBytes 5.13 Mbits/sec 0 92.6 KBytes
[ 5] 3.00-4.00 sec 626 KBytes 5.13 Mbits/sec 4 72.2 KBytes
[ 5] 4.00-5.00 sec 626 KBytes 5.13 Mbits/sec 0 79.0 KBytes
[ 5] 5.00-6.00 sec 626 KBytes 5.13 Mbits/sec 0 80.3 KBytes
[ 5] 6.00-7.00 sec 626 KBytes 5.13 Mbits/sec 0 85.8 KBytes
[ 5] 7.00-8.00 sec 626 KBytes 5.13 Mbits/sec 0 91.2 KBytes
[ 5] 8.00-9.00 sec 599 KBytes 4.91 Mbits/sec 2 74.9 KBytes
[ 5] 9.00-10.00 sec 626 KBytes 5.13 Mbits/sec 0 85.8 KBytes
[ 5] 10.00-10.05 sec 0.00 Bytes 0.00 bits/sec 0 87.1 KBytes

[ ID] Interval Transfer Bandwidth Retr [ 5] 0.00-10.05 sec 6.31 MBytes 5.27 Mbits/sec 15 sender [ 5] 0.00-10.05 sec 0.00 Bytes 0.00 bits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 69.4 KBytes 569 Kbits/sec
[ 5] 1.00-2.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 2.00-3.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 3.00-4.00 sec 77.6 KBytes 635 Kbits/sec
[ 5] 4.00-5.00 sec 79.0 KBytes 647 Kbits/sec
[ 5] 5.00-6.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 6.00-7.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 7.00-8.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 8.00-9.00 sec 74.9 KBytes 613 Kbits/sec
[ 5] 9.00-10.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 10.00-10.11 sec 8.17 KBytes 595 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.11 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.11 sec 772 KBytes 625 Kbits/sec receiver

    WAN3

path UP status: OK

mtu: 1418 bytes rtt: 32.753 ms rttvar: 0.536 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1165269 packets total rx: 992853 packets

iPerf Download:

Reverse mode, remote host 172.81.182.185 is sending [ 5] local 192.168.12.2 port 62258 connected to 172.81.182.185 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 634 KBytes 5.20 Mbits/sec
[ 5] 1.00-2.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 2.00-3.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 3.00-4.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 4.00-5.00 sec 622 KBytes 5.10 Mbits/sec
[ 5] 5.00-6.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 6.00-7.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 7.00-8.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 8.00-9.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 9.00-10.00 sec 619 KBytes 5.07 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.33 MBytes 5.31 Mbits/sec 11 sender [ 5] 0.00-10.00 sec 6.07 MBytes 5.09 Mbits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 1.00-2.00 sec 74.9 KBytes 613 Kbits/sec
[ 5] 2.00-3.00 sec 83.0 KBytes 680 Kbits/sec
[ 5] 3.00-4.00 sec 81.7 KBytes 669 Kbits/sec
[ 5] 4.00-5.00 sec 83.0 KBytes 680 Kbits/sec
[ 5] 5.00-6.00 sec 81.7 KBytes 669 Kbits/sec
[ 5] 6.00-7.00 sec 81.7 KBytes 669 Kbits/sec
[ 5] 7.00-8.00 sec 81.7 KBytes 669 Kbits/sec
[ 5] 8.00-9.00 sec 83.0 KBytes 680 Kbits/sec
[ 5] 9.00-10.00 sec 83.0 KBytes 680 Kbits/sec
[ 5] 10.00-10.09 sec 8.17 KBytes 728 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.09 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.09 sec 822 KBytes 667 Kbits/sec receiver

    WAN4

path UP status: OK

mtu: 1418 bytes rtt: 30.477 ms rttvar: 1.319 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1169247 packets total rx: 995681 packets

iPerf Download: Reverse mode, remote host 172.81.182.185 is sending [ 5] local 192.168.13.2 port 46222 connected to 172.81.182.185 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 641 KBytes 5.25 Mbits/sec
[ 5] 1.00-2.00 sec 621 KBytes 5.08 Mbits/sec
[ 5] 2.00-3.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 3.00-4.00 sec 381 KBytes 3.12 Mbits/sec
[ 5] 4.00-5.00 sec 858 KBytes 7.03 Mbits/sec
[ 5] 5.00-6.00 sec 623 KBytes 5.11 Mbits/sec
[ 5] 6.00-7.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 7.00-8.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 8.00-9.00 sec 622 KBytes 5.10 Mbits/sec
[ 5] 9.00-10.00 sec 619 KBytes 5.07 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.58 MBytes 5.52 Mbits/sec 12 sender [ 5] 0.00-10.00 sec 6.08 MBytes 5.10 Mbits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 1.00-2.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 2.00-3.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 3.00-4.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 4.00-5.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 5.00-6.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 6.00-7.00 sec 79.0 KBytes 647 Kbits/sec
[ 5] 7.00-8.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 8.00-9.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 9.00-10.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 10.00-10.12 sec 9.53 KBytes 636 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.12 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.12 sec 787 KBytes 637 Kbits/sec receiver

    WAN5

Glorytun:

path UP status: OK mtu: 1418 bytes rtt: 30.502 ms rttvar: 0.497 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1166478 packets total rx: 995404 packets

iPerf Download: Reverse mode, remote host 172.81.182.185 is sending [ 5] local 192.168.14.2 port 54984 connected to 172.81.182.185 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 659 KBytes 5.40 Mbits/sec
[ 5] 1.00-2.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 2.00-3.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 3.00-4.00 sec 617 KBytes 5.05 Mbits/sec
[ 5] 4.00-5.00 sec 618 KBytes 5.06 Mbits/sec
[ 5] 5.00-6.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 6.00-7.00 sec 475 KBytes 3.89 Mbits/sec
[ 5] 7.00-8.00 sec 765 KBytes 6.27 Mbits/sec
[ 5] 8.00-9.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 9.00-10.00 sec 622 KBytes 5.10 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.60 MBytes 5.54 Mbits/sec 14 sender [ 5] 0.00-10.00 sec 6.09 MBytes 5.11 Mbits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 73.5 KBytes 602 Kbits/sec
[ 5] 1.00-2.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 2.00-3.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 3.00-4.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 4.00-5.00 sec 70.8 KBytes 580 Kbits/sec
[ 5] 5.00-6.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 6.00-7.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 7.00-8.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 8.00-9.00 sec 73.5 KBytes 602 Kbits/sec
[ 5] 9.00-10.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 10.00-10.09 sec 6.81 KBytes 597 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.09 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.09 sec 766 KBytes 622 Kbits/sec receiver

iPerf Download:

    WAN6

path UP status: OK

mtu: 1418 bytes rtt: 31.646 ms rttvar: 0.545 ms rate tx: 800000 bytes/sec rate rx: 6000000 bytes/sec total tx: 1166190 packets total rx: 993286 packets

iPerf Download:

Reverse mode, remote host 172.81.182.185 is sending [ 5] local 192.168.15.2 port 51882 connected to 172.81.182.185 port 5201 [ ID] Interval Transfer Bitrate [ 5] 0.00-1.00 sec 637 KBytes 5.22 Mbits/sec
[ 5] 1.00-2.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 2.00-3.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 3.00-4.00 sec 530 KBytes 4.34 Mbits/sec
[ 5] 4.00-5.00 sec 709 KBytes 5.81 Mbits/sec
[ 5] 5.00-6.00 sec 619 KBytes 5.07 Mbits/sec
[ 5] 6.00-7.00 sec 535 KBytes 4.38 Mbits/sec
[ 5] 7.00-8.00 sec 705 KBytes 5.78 Mbits/sec
[ 5] 8.00-9.00 sec 621 KBytes 5.09 Mbits/sec
[ 5] 9.00-10.00 sec 622 KBytes 5.10 Mbits/sec

[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 6.49 MBytes 5.44 Mbits/sec 17 sender [ 5] 0.00-10.00 sec 6.07 MBytes 5.09 Mbits/sec receiver

iPerf Upload:

[ ID] Interval Transfer Bandwidth [ 5] 0.00-1.00 sec 79.0 KBytes 647 Kbits/sec
[ 5] 1.00-2.00 sec 74.9 KBytes 613 Kbits/sec
[ 5] 2.00-3.00 sec 80.3 KBytes 658 Kbits/sec
[ 5] 3.00-4.00 sec 74.9 KBytes 613 Kbits/sec
[ 5] 4.00-5.00 sec 76.2 KBytes 624 Kbits/sec
[ 5] 5.00-6.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 6.00-7.00 sec 77.6 KBytes 636 Kbits/sec
[ 5] 7.00-8.00 sec 79.0 KBytes 647 Kbits/sec
[ 5] 8.00-9.00 sec 76.2 KBytes 625 Kbits/sec
[ 5] 9.00-10.00 sec 73.5 KBytes 602 Kbits/sec
[ 5] 10.00-10.10 sec 8.17 KBytes 641 Kbits/sec

[ ID] Interval Transfer Bandwidth [ 5] 0.00-10.10 sec 0.00 Bytes 0.00 bits/sec sender [ 5] 0.00-10.10 sec 777 KBytes 630 Kbits/sec receiver

user747 commented 5 years ago

Sorry forgot to include client bench

bench chacha20poly1305 libsodium 1.0.18

precision 2^(-10) bufsize 65536 bytes duration 1 second

2^n min avg max delta

0 4.26 Mbps 4.32 Mbps 4.32 Mbps 8.70e-04 1 8.66 Mbps 8.66 Mbps 8.70 Mbps 5.69e-04 2 17.19 Mbps 17.31 Mbps 17.31 Mbps 2.46e-03 3 34.43 Mbps 34.43 Mbps 34.52 Mbps 2.01e-02 4 67.61 Mbps 67.61 Mbps 67.67 Mbps 5.28e-02 5 127.21 Mbps 127.21 Mbps 127.27 Mbps 5.30e-02 6 249.37 Mbps 251.82 Mbps 251.82 Mbps 2.13e-01 7 362.69 Mbps 362.69 Mbps 362.92 Mbps 2.33e-01 8 480.22 Mbps 480.28 Mbps 480.28 Mbps 5.86e-02 9 578.28 Mbps 578.28 Mbps 578.36 Mbps 8.48e-02 10 643.93 Mbps 643.98 Mbps 643.98 Mbps 4.92e-02 11 680.47 Mbps 681.48 Mbps 682.95 Mbps 1.97e-01 12 704.32 Mbps 704.32 Mbps 704.44 Mbps 1.23e-01 13 712.61 Mbps 714.29 Mbps 714.29 Mbps 4.10e-01 14 719.84 Mbps 720.20 Mbps 720.83 Mbps 3.62e-01 15 723.21 Mbps 723.52 Mbps 723.52 Mbps 3.07e-01 16 725.05 Mbps 725.05 Mbps 725.49 Mbps 4.43e-01

angt commented 5 years ago

Hi, Happy to see it works well on your setup :) As you said there is a tunnel overhead and I think we can't do much better. Failover also have a cost, I could add an option to make it less impacting.

user747 commented 5 years ago

Failover also have a cost, I could add an option to make it less impacting.

Yes that would be interesting feature to test.