L4STeam / linux

Kernel tree containing patches for TCP Prague and the dualpi2 qdisc
Other
47 stars 16 forks source link

Prague underperforming in congestion scenario #25

Closed MatteoGuarna closed 8 months ago

MatteoGuarna commented 10 months ago

Greetings,

Apologize for my second issue: I am still trying to perform some fairness measurements with both L4S and classic flow. I now though migrated to a physical environment and I seem not to be able to achieve the correct configuration.

The setup is very simple: I have four servers (two senders and two receivers) exchanging two traffic flows through one server router. One client-server pair uses Prague as CC, while the other uses Cubic. All servers have the patched kernel provided in this repository branch.

Still, if I trigger a congestion by generating both the Prague and the Cubic flows (let's say the flows measure 100 Mbit/s each, and they come though a L2 switch both on the same router's input interface on a 1Gb Ethernet link; only a 100M link though is in place on the output interface towards the receivers) I see the L4S flow having higher delay, higher jitter and a smaller (and more variable) bandwidth share. The Prague share is 1/4 of the Cubic share.

I configured my L4S endpoints as follows:

I configured my router server as follows:

I then applied the fair queue and disabled the offloading capabilities on both my classic endpoints to keep the classic and l4s flows as similar as possible, but to no avail (even without these precautions the results remain roughly the same).

I am sure I am missing some important details in the setup, and I would really appreciate some help.

Regards,

Matteo

P.s. I just want to point out that by looking at the packet traces everything seems fine: Prague carries the ECN=1, the dualpi2 marks packets with ECN=3, the AccEcn control signals on the ACE fields are coherent, and no losses occur in the Prague flow, when they do happen with Cubic... It just looks like Prague is underperforming for whatever reason. Furthermore, if I switch back to two Cubic flows I measure perfect share, equal delay and equal jitter, so it looks to me like there are no physical impairments on the testbed.

MatteoGuarna commented 8 months ago

If anyone's interested, for what is worth I was able to solve this issue. I found this solution almost by accident while trying to replicate the same scenario using Linux namespaces, and it turned out that the issue is with dualpi2.

I found out that in order to ensure that the L4S Dual Queue (dualpi2) works correctly, the routers' interfaces where dualpi2 is deployed need not only to have dualpi2 enabled with traffic control, but also tbf (any reasonable set of parameters should work, according to your needs). This was the only solution I found to ensure that Dual Queue is not starving the Prague flows. For example:

sudo tc qdisc add dev $INPUT_IF root handle 1: tbf rate 1000mbps latency 10ms burst 1540
#sudo tc qdisc add dev $INPUT_IF parent 1: dualpi2
#the modifications must be added to the router's input as well as output interface
sudo tc qdisc add dev $OUTPUT_IF root handle 1: tbf rate 1000mbps latency 10ms burst 1540
sudo tc qdisc add dev $OUTPUT_IF parent 1: dualpi2