QUIC+TCP Backpressure Issue

[heixiaoma]

Background:

• Client sends data to the QUIC Server via QUIC protocol: The client sends data to the QUIC Server using the QUIC protocol. QUIC is a transport protocol based on UDP, and data is transmitted in the form of packets.

• QUIC Server forwards data to the TCP Server: Once the QUIC Server receives the data, it forwards it to a backend TCP Server. Due to the flow control mechanisms of the TCP protocol, backpressure may occur, where the processing speed of the TCP Server cannot keep up with the data sending speed of the QUIC Server, leading to data backlog.

Current approach:

• Disabling automatic reading (AUTO_READ=false): In the QUIC Server, the setting AUTO_READ=false is used to disable the automatic read behavior of the QUIC protocol. Then, in the request handler function of the QUIC Server, new data is manually read from the QUIC StreamChannel each time data is successfully sent to the TCP Server.

Issues encountered:

• Risk of memory leak: Even though manual reading is used, there is still a risk of memory leaks. Data that is read might not be properly released, leading to accumulated data in memory.

• Differences between UDP and TCP: Since QUIC is based on the UDP protocol, and TCP has fundamentally different flow control and congestion control mechanisms, a mismatch occurs between the flow control mechanisms of QUIC and TCP when manually reading data. Specifically:

• • UDP handles data by packets: UDP is a packet-based transmission protocol, where each unit of data sent is a packet, rather than a data stream. This means that the amount of data read by the QUIC Server from the UDP layer at a time may be large, and there is no flow control mechanism like in TCP to limit the amount of data read at once.
• • TCP handles data by streams: TCP, on the other hand, is stream-based, with built-in flow control and congestion control mechanisms. The receiving rate of the TCP Server can be limited by flow control, meaning that it might not be able to keep up with the sending rate of the QUIC Server, resulting in backpressure.

• Data backlog issue: Since UDP transmits data in packets, each time QUIC Server reads data, the amount of data it reads might exceed the processing capacity of the TCP Server. This leads to data backlog at the QUIC Server, which is faster than the TCP Server's processing speed. As a result, even though manual data reading is implemented on the QUIC Server side, the data backlog cannot be effectively mitigated.

The issue has been bothering me for a long time, and I want to discuss how to handle it. Whenever TCP processing is slow, there is a backlog in the QUIC server

Even with the use of channelWritabilityChanged and high/low watermarks to handle the issue, the problem still persists.

@normanmaurer Hello, expert. Sorry to bother you, but could I get your thoughts and advice?

[heixiaoma]

The TCP bandwidth is 3 Mbps, and the QUIC bandwidth is 100 Mbps.