Request for Guidance: Implementing an Abstract Transport Layer to Support Both TCP/UDP and QUIC

[anchalshivank]

Overview

I am interested in extending rqbit to support both TCP/UDP and QUIC as potential transport protocols. Currently, rqbit operates over TCP/UDP, which aligns with the standard BitTorrent protocol, but adding QUIC support could bring several benefits:

• Improved Performance: QUIC offers faster connection setup and built-in encryption.
• Modern Transport Capabilities: QUIC provides multiplexed streams and better handling of network congestion.

Goal

The goal is to introduce an abstract transport layer within rqbit that allows interchangeable use of TCP/UDP or QUIC. This would enable rqbit to operate over traditional BitTorrent networks while also providing the option for a QUIC-based setup.

Request for Guidance

Since I’m new to working with rqbit's codebase, I’d appreciate any guidance on how to approach this. Specifically:

1. Key Components to Modify:

   • Which modules or files in rqbit are responsible for TCP/UDP handling?
   • Are there specific functions or abstractions already in place that could help with creating a transport layer?

2. Recommended Approach for Abstraction:

   • Should I create a trait (e.g., Transport) to define basic operations like connect, send, receive, and close?
   • Is there an existing pattern within rqbit for handling multiple protocols that could be extended to support both TCP/UDP and QUIC?

3. Potential Challenges:

   • Are there known issues or challenges with adapting rqbit for non-standard transports like QUIC?
   • Would supporting QUIC potentially interfere with the BitTorrent protocol in a way that might impact compatibility?

4. Implementation of QUIC:

   • For implementing the QUIC transport, would you recommend using a library like quinn, or are there other options that might be better suited for integration with rqbit?

Additional Context

I believe this enhancement could make rqbit more adaptable to modern network environments, where QUIC is becoming increasingly common. However, I’d like to ensure I follow the best approach to keep rqbit's structure clean and maintainable.

[ikatson]

BitTorrent doesn't work over QUIC, why would you want to add support for it?

As for abstracting away transport, it already is for stream protocols (TCP, TCP over SOCKS5). It was done just for socks, but extending to support smth like QUIC should be very easy - all you need is implement connect(), AsyncRead and AsyncWrite.

But the real benefit for rqbit would be to add support for uTP, but the current abstraction isn't enough for it - you need to implement congestion control and other parts of uTP.

[radumarias]

I think there are advantages in using QUIC

μTP Does not natively include encryption. Typically relies on the application layer or external protocols like TLS for security.

Congestion control algorithms are more advanced in QUIC.

μTP Optimized for low latency in congested networks. May experience higher overhead compared to UDP when prioritizing fairness.

Other p2p solutions are using QUIC like Iroh and lubp2p.

[radumarias]

We might try to support μTP too and have some benchmarks.

[radumarias]

you need to implement congestion control and other parts of uTP.

@ikatson why is that? as μTP already supports that.

[ikatson]

I don't understand what rqbit (a torrent client!) has to do with QUIC.

Sure you can add QUIC but what's the point if the bit torrent network doesn't support it?

If it supported QUIC there would be a design doc (BEP) that would explain how does it integrate into existing network.

[radumarias]

@ikatson I think the original question was not formulated ok. We want to add this extension for a cluster that is used in our case only internally and we don't require compatibility with default BitTorrent external clients.

We are working on https://github.com/radumarias/rfs and we want to use BitTorrent to sync files between our internal nodes. So our plan is to have a BitTorrent custom client + protocol to make the transfer over UDP, again, just between our internal nodes, this will not communicate with other BitTorrent clients.

You mentioned above that QUIC would be easy to integrate, this is good for us then. But you also mentioned that μTP would be harder to add because the abstractizarion is not yet prepared. Given these 2 are very similar, they are both over UDP, handle congestion control and retries, can you explain please why is that the case?

[radumarias]

Given uTP is in the BitTorrent protocol we would like to add it to rqbit because it will benefit to be able to communicate with other standard implementation clients which have this. And it will fallback to TCP with clients that don't support it. Nowadays most of the BitTorrent clients supports it.

But can you tell us why it would be harder to add compared to QUIC and can you give us some guidance on how to add it?

The support for QUIC we will add just in out internal fork just for internal use. We olan to extend that a bit more and use Iroh actually, not even QUIC directly, because Iroh has an internal DHT which we need because we need internal one in our case, we can't use public DHT and trackers as files need to remain private in our cluster.

Another approach for us, which makes more sense now, is to first add support for uTP, as our initial goal was to have BitTorrent over UDP, we didn't know that uTP is actually in the protocol, if we did this would gave been out first approach, and we just use Iroh for DHT.

[ikatson]

I was wrong - it should be similar in effort. I glossed over BEP 29 (uTP) to double-check - in my memory it changed the operation of the BitTorrent protocol itself, so leaked from transport to application layer. But it doesn't seem to be the case.

So as long as this transport can be made AsyncRead + AsyncWrite - the changes should be minimal. You just change this "connect() -> Box" to return smth different than it does, and none of the internals (which operate over bittorrent application layer) will need to be changed.