Scheduler: add redundant scheduler support in BPF

[vempire-ghost]

I currently use mptcp out-of-tree because for what I need, I have to use the redundant scheduler as I require connection resilience rather than speed increase. However, up to now, the upstream version doesn't have this scheduler, and I haven't found information that a redundant scheduler is being developed. I would like to know if there are plans to implement a redundant scheduler in this version of mptcp?

[mjmartineau]

The plan is to address other packet scheduling algorithms with the BPF packet scheduler (#75).

Supporting redundant schedulers has been one of the challenges with getting the BPF scheduler fully implemented, and we haven't been sure how much to prioritize redundant scheduler capabilities in the scheduler framework. Could you comment with more details about your use case for redundant schedulers, either here or in #75?

[vempire-ghost]

Thank you very much for your response. I use the redundant mode to create a highly resilient proxy/VPN for programs with low or no tolerance for packet loss, such as online games, for example. In the default scheduler for both mptcp v0 and v1, it takes too long to redirect traffic to another subflow if the current one is experiencing issues. In a program where fault tolerance is zero and ideal latency is less than 30ms, this causes performance problems. However, using the redundant mode of mptcp v0 solves this, as each packet now has several copies of itself going through different connections, ensuring a high chance that at least one of them reaches the destination. I hope I was able to explain, and if you need more information, I'm available. Once again, thank you very much; your work is fantastic.

[matttbe]

I use the redundant mode to create a highly resilient proxy/VPN for programs with low or no tolerance for packet loss, such as online games, for example. In the default scheduler for both mptcp v0 and v1, it takes too long to redirect traffic to another subflow if the current one is experiencing issues. In a program where fault tolerance is zero and ideal latency is less than 30ms, this causes performance problems. However, using the redundant mode of mptcp v0 solves this, as each packet now has several copies of itself going through different connections, ensuring a high chance that at least one of them reaches the destination.

Thank you for explaining the use-case. To be honest, I don't think using MPTCP for this use-case is a good idea: each subflow is a valid TCP connection where lost packets have to be re-transmitted. In other words, if a packet is lost, this will cause some delays on that path because this packet will need to be re-transmitted (even if it has been received via another subflow) before moving on. Without that, some middleboxes might block some subflows.

As a workaround, we could have the PM recreating a subflow when there are some losses (and other subflows available).

Maybe for this specific use-case, the best is to use a tunnel (not using TCP) where frames would be duplicated over multiple paths.

Notes:

• You can also tweak net.mptcp.stale_loss_cnt sysctl value (it makes sense if some subflows have the backup flag).
• "MPTCP v0" refers to the protocol version, not the implementation (some supports multiple versions), see https://github.com/multipath-tcp/mptcp_net-next/wiki#upstream-vs-out-of-tree-implementations

[vempire-ghost]

I use the redundant mode to create a highly resilient proxy/VPN for programs with low or no tolerance for packet loss, such as online games, for example. In the default scheduler for both mptcp v0 and v1, it takes too long to redirect traffic to another subflow if the current one is experiencing issues. In a program where fault tolerance is zero and ideal latency is less than 30ms, this causes performance problems. However, using the redundant mode of mptcp v0 solves this, as each packet now has several copies of itself going through different connections, ensuring a high chance that at least one of them reaches the destination.

Thank you for explaining the use-case. To be honest, I don't think using MPTCP for this use-case is a good idea: each subflow is a valid TCP connection where lost packets have to be re-transmitted. In other words, if a packet is lost, this will cause some delays on that path because this packet will need to be re-transmitted (even if it has been received via another subflow) before moving on. Without that, some middleboxes might block some subflows.

As a workaround, we could have the PM recreating a subflow when there are some losses (and other subflows available).

Maybe for this specific use-case, the best is to use a tunnel (not using TCP) where frames would be duplicated over multiple paths.

Notes:

* You can also tweak [`net.mptcp.stale_loss_cnt` sysctl value](https://docs.kernel.org/networking/mptcp-sysctl.html) (it makes sense if some subflows have the backup flag).

* "MPTCP v0" refers to the protocol version, not the implementation (some supports multiple versions), see https://github.com/multipath-tcp/mptcp_net-next/wiki#upstream-vs-out-of-tree-implementations

Thank you very much for the response, matttbe. I indeed expressed myself poorly; when I referred to v0, I meant the out-of-the-tree implementation of mptcp. For years, I've been looking for solutions to this issue, and I thought mptcp would be the most viable. I've tested other proprietary solutions like Speedify, but their servers leave much to be desired, and not being able to use my own exclusive server is something I wouldn't want to give up.

Regarding the mentioned tunnel, I didn't quite understand what kind of program could achieve what you described. Could you elaborate more?

Finally, on the website https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP, it says about the redundant scheduler, 'redundant': This scheduler will try to transmit traffic on all available subflows in a redundant way. It is useful when one wants to achieve the lowest possible latency by sacrificing bandwidth." This led me to interpret that in this mode, the packet would be sent simultaneously on all subflows, and whatever arrives first would be forwarded, with the rest discarded. In my practical tests, I intentionally disconnect or degrade while monitoring in real-time the in-game latency, and there is no loss even when disconnecting cables and reconnecting them afterward using the redundant scheduler and the wvegas congestion control.

[matttbe]

Regarding the mentioned tunnel, I didn't quite understand what kind of program could achieve what you described. Could you elaborate more?

I don't know if such a solution exists but it should not be complicated to put in place I think: using an existing tunnel solution (VPN), everything is sent over all the different paths between two hosts, and what is duplicated is dropped.

Finally, on the website https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP, it says about the redundant scheduler, 'redundant': This scheduler will try to transmit traffic on all available subflows in a redundant way. It is useful when one wants to achieve the lowest possible latency by sacrificing bandwidth." This led me to interpret that in this mode, the packet would be sent simultaneously on all subflows, and whatever arrives first would be forwarded, with the rest discarded.

That's correct and I understand it can work well in some conditions but not if the paths are "lossy" and/of if the bandwidth is "low enough".

So even if it is not the perfect solution for this use case, I understand MPTCP can help. In other words, I'm not against having such a packet scheduler if someone is whiling to implement it. But that's not a priority and the changes in the core to have it should not impact the other use-cases.

Let me change the title of this GH issue.

[vempire-ghost]

Regarding the mentioned tunnel, I didn't quite understand what kind of program could achieve what you described. Could you elaborate more?

I don't know if such a solution exists but it should not be complicated to put in place I think: using an existing tunnel solution (VPN), everything is sent over all the different paths between two hosts, and what is duplicated is dropped.

Finally, on the website https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP, it says about the redundant scheduler, 'redundant': This scheduler will try to transmit traffic on all available subflows in a redundant way. It is useful when one wants to achieve the lowest possible latency by sacrificing bandwidth." This led me to interpret that in this mode, the packet would be sent simultaneously on all subflows, and whatever arrives first would be forwarded, with the rest discarded.

That's correct and I understand it can work well in some conditions but not if the paths are "lossy" and/of if the bandwidth is "low enough".

So even if it is not the perfect solution for this use case, I understand MPTCP can help. In other words, I'm not against having such a packet scheduler if someone is whiling to implement it. But that's not a priority and the changes in the core to have it should not impact the other use-cases.

Let me change the title of this GH issue.

Every VPN, SSH tunnel, or proxy I tried operates with TCP or UDP, which are necessarily restricted to a single path. That's why the mptcp project caught my attention, and I started using it, and it was the only non-proprietary one that could do what I was looking for. I would be happy if you could add the redundant scheduler to the upstream, just like it is in the out-of-the-tree version. If you need more information, I'm happy to help, and I appreciate your work on this project once again.