quic transport support

[juzi5201314]

quic

I tried to achieve a simple implementation... #64

[rapiz1]

QUIC sounds good and your initial work is awesome.

However, I'm on the fence about adding QUIC since TLS on TCP is already supported. Adding an alternative is tempting but it should be a useful addition, instead of adding up maintenance burden. I have some questions:

• Are there any use cases that QUIC must be used and can't be replaced by TLS on TCP?
• Or, are there some situations that QUIC is significant better in terms of latency, performance, etc. ? If so, must it be implemented in rathole itself? Can using a QUIC tunnel tool with rathole, achieve similar performance/latency?

I'm glad to hear some user feedback here if anyone is interested in using QUIC.

[juzi5201314]

I tried to do some benchmarks, and the noise pattern is the default value.

Single

cmd: iperf3 -c 127.0.0.1 -p 5203 -Z -u -b 0:

quic:

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  4.75 GBytes  4.08 Gbits/sec  0.000 ms  0/3524250 (0%)  sender
[  5]   0.00-10.00  sec   900 MBytes   755 Mbits/sec  0.009 ms  2862089/3513774 (81%)  receiver

noise:

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  4.92 GBytes  4.23 Gbits/sec  0.000 ms  0/3650140 (0%)  sender
[  5]   0.00-10.00  sec   557 MBytes   467 Mbits/sec  0.021 ms  3219014/3622310 (89%)  receiver

Parallel(10)

cmd: iperf3 -c 127.0.0.1 -p 5203 -Z -u -b 0 -P 10

quic:

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[  5]   0.00-10.00  sec   117 MBytes  98.5 Mbits/sec  0.132 ms  106009/191061 (55%)  receiver
[  7]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[  7]   0.00-10.00  sec   117 MBytes  98.5 Mbits/sec  0.127 ms  106032/191061 (55%)  receiver
[  9]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[  9]   0.00-10.00  sec   117 MBytes  98.5 Mbits/sec  0.145 ms  106004/191061 (55%)  receiver
[ 11]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 11]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.143 ms  106068/191061 (56%)  receiver
[ 13]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 13]   0.00-10.00  sec   117 MBytes  98.3 Mbits/sec  0.141 ms  106165/191061 (56%)  receiver
[ 15]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 15]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.137 ms  106123/191056 (56%)  receiver
[ 17]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 17]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.138 ms  106104/191056 (56%)  receiver
[ 19]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 19]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.139 ms  106062/191056 (56%)  receiver
[ 21]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 21]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.134 ms  106064/191056 (56%)  receiver
[ 23]   0.00-10.00  sec   264 MBytes   221 Mbits/sec  0.000 ms  0/191180 (0%)  sender
[ 23]   0.00-10.00  sec   117 MBytes  98.4 Mbits/sec  0.137 ms  106114/191060 (56%)  receiver
[SUM]   0.00-10.00  sec  2.58 GBytes  2.21 Gbits/sec  0.000 ms  0/1911800 (0%)  sender
[SUM]   0.00-10.00  sec  1.15 GBytes   984 Mbits/sec  0.137 ms  1060745/1910589 (55%)  receiver

noise:

[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[  5]   0.00-10.00  sec  55.1 MBytes  46.2 Mbits/sec  0.205 ms  168434/208302 (81%)  receiver
[  7]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[  7]   0.00-10.00  sec  55.1 MBytes  46.2 Mbits/sec  0.217 ms  168388/208302 (81%)  receiver
[  9]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[  9]   0.00-10.00  sec  55.0 MBytes  46.1 Mbits/sec  0.178 ms  168474/208302 (81%)  receiver
[ 11]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 11]   0.00-10.00  sec  55.0 MBytes  46.2 Mbits/sec  0.190 ms  168445/208301 (81%)  receiver
[ 13]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 13]   0.00-10.00  sec  54.6 MBytes  45.8 Mbits/sec  0.179 ms  168734/208301 (81%)  receiver
[ 15]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 15]   0.00-10.00  sec  54.6 MBytes  45.8 Mbits/sec  0.199 ms  168785/208301 (81%)  receiver
[ 17]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 17]   0.00-10.00  sec  54.7 MBytes  45.9 Mbits/sec  0.181 ms  168664/208301 (81%)  receiver
[ 19]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 19]   0.00-10.00  sec  54.7 MBytes  45.9 Mbits/sec  0.201 ms  168676/208301 (81%)  receiver
[ 21]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 21]   0.00-10.00  sec  54.7 MBytes  45.9 Mbits/sec  0.206 ms  168667/208301 (81%)  receiver
[ 23]   0.00-10.00  sec   289 MBytes   243 Mbits/sec  0.000 ms  0/209470 (0%)  sender
[ 23]   0.00-10.00  sec  54.7 MBytes  45.9 Mbits/sec  0.200 ms  168679/208300 (81%)  receiver
[SUM]   0.00-10.00  sec  2.82 GBytes  2.43 Gbits/sec  0.000 ms  0/2094700 (0%)  sender
[SUM]   0.00-10.00  sec   548 MBytes   460 Mbits/sec  0.195 ms  1685946/2083012 (80%)  receiver

on: ubuntu20-wsl2 mtu 1500

[juzi5201314]

From the simple benchmark above, it seems that quic has lower jitter and packet loss rate.

[juzi5201314]

From the simple benchmark above, it seems that quic has lower jitter and packet loss rate.

However, when dealing with tcp instead of udp, the bit rate of quic is lower than that of noise.

[rapiz1]

Good to see you've done the benchmark. But what's important is that, when different transport protocol are compared, local loopback tests can barely reveal how they performance in the real world because it omits lots of factors, like the congestion control, the packet loss, etc.. Local loopback is only useful when you want to compare different implementations of the same protocol.

[rapiz1]

In another word, the real world workload matters here. That's why I want to hear more user feedback in terms of adding QUIC.

[juzi5201314]

Yes, according to the characteristics of quic, it should be better than tcp when the network is relatively poor. (As far as I know, many people in our country use frp to map home servers to foreign servers, so there are often network conditions that are not Ideal). Of course, in any case, the reality requires user feedback (:

Maybe I can use tc to simulate the real bad network conditions to a certain extent?

[anderspitman]

In theory, the key performance advantage of QUIC is removal of head-of-line blocking. Basically, in a lossy network, when you have multiple streams multiplexed over a single TCP connection (such as HTTP/2), if you lose packets on one stream, all the other streams block until those packets are re-transmitted. Since QUIC uses UDP, there is no dependency between streams so this problem goes away.

I'm not sure how big of an issue this is in practice, since it seems to me that if you're losing packets on one stream, you're probably losing packets on most of them. But I have heard that in certain cases in lossy networks HTTP/1.1 can have better performance than HTTP/2 because HTTP/1.1 typically uses a pool of TCP connections.

[rapiz1]

Maybe I can use tc to simulate the real bad network conditions to a certain extent?

That will be great! @juzi5201314

[bhzhu203]

How about add the KCP support to rathole @rapiz1

[aa51513]

How about add the KCP support to rathole @rapiz1

I'm also very much looking forward to this, if it will be considered

[firecow]

Cloudflare tunnels recently switched to QUIC as their default transport.

Their argument basicly was less congestion if the big bad internet looses a packet.

In cases, where rathole is used over the internet, QUIC would be rather awesome.

[anderspitman]

frp also supports QUIC now: https://github.com/fatedier/frp/pull/3198

[firecow]

https://github.com/cloudflare/quiche Cloudflare open sourced their QUIC implementation btw, perhaps that can be used somehow.