libp2p / go-libp2p

libp2p implementation in Go
MIT License
6.09k stars 1.07k forks source link

Can quic.Transport be directly connected? #2649

Open peakedshout opened 12 months ago

peakedshout commented 12 months ago

Hey, I'm new to this thing and I'm trying to make a connection with p2p via quic, exchanging addresses through a relay server. I'm having trouble: after establishing a standard quic connection to the relay server, I can't establish a connection through libp2p? (Maybe I just didn't know it), on tcp, I can enable port multiplexing to make two connections to one port. But in quic, it seems that if I set up multiplexing, it still tells me that the port is already in use. I am confused and want to be able to get one through quic.Transport inheriting the same udpconn

Jorropo commented 12 months ago

Assuming "port multiplexing" means reuseport, QUIC default to this, once you have one transport you can reuse it for multiple connections and listens by calling .Dial and .Listen multiple times on the same transport (you might only be able to listen once but then .Accept can be called in a loop).

However I don't belive it is intended to call the QUIC transport yourself directly, you should use the magic config package: github.com/libp2p/go-libp2p, you can checkout the examples in https://github.com/libp2p/go-libp2p/tree/master/examples

peakedshout commented 12 months ago

Assuming "port multiplexing" means reuseport, QUIC default to this, once you have one transport you can reuse it for multiple connections and listens by calling .Dial and .Listen multiple times on the same transport (you might only be able to listen once but then .Accept can be called in a loop).

However I don't belive it is intended to call the QUIC transport yourself directly, you should use the magic config package: github.com/libp2p/go-libp2p, you can checkout the examples in https://github.com/libp2p/go-libp2p/tree/master/examples

I think I didn't express it clearly enough. I have a quic.Transport externally, but I can't pass it into libp2p. When I look at the examples, quic.Transport is built through libp2p. Obviously, my communication with the relay server is not through libp2p, and the quic.Transport obtained cannot be reused. If possible, could you tell me how to include quic.Transport?

Jorropo commented 12 months ago

Is quic.Transport a quic-go or go-libp2p object ? Like this: https://pkg.go.dev/github.com/quic-go/quic-go#Transport Or like this: https://github.com/libp2p/go-libp2p/blob/abca4e693014a6c64a5c66790a742affca0700ef/p2p/transport/quic/transport.go#L38

peakedshout commented 12 months ago

Is quic.Transport a quic-go or go-libp2p object ? Like this: https://pkg.go.dev/github.com/quic-go/quic-go#Transport Or like this:

https://github.com/libp2p/go-libp2p/blob/abca4e693014a6c64a5c66790a742affca0700ef/p2p/transport/quic/transport.go#L38

Apparently the former, the transport of the latter was not exposed. I establish communication with the relay server using quic-go

Jorropo commented 12 months ago

I'm missing something, why are you mixing Pure-QUIC and Libp2p-QUIC connections in a same app ? Not that this is wrong or anything but it sounds harder to manage.


So if I understand your problem correctly, you want to reuse the same quic.Transport object inside libp2p and outside libp2p for your own Pure-QUIC connections ?

I don't see any practical issue doing this, as long as you don't mind the listener (you can't easily share the listener because except maybe ALPN I don't see how you would route different connections to the right part of your application (libp2p or your custom thing), however with webtransport the HTTP alpn is used by libp2p too).

peakedshout commented 12 months ago

I'm missing something, why are you mixing Pure-QUIC and Libp2p-QUIC connections in a same app ? Not that this is wrong or anything but it sounds harder to manage.

So if I understand your problem correctly, you want to reuse the same quic.Transport object inside libp2p and outside libp2p for your own Pure-QUIC connections ?

I don't see any practical issue doing this, as long as you don't mind the listener (you can't easily share the listener because except maybe ALPN I don't see how you would route different connections to the right part of your application (libp2p or your custom thing), however with webtransport the HTTP alpn is used by libp2p too).

This is not difficult to manage, the relay server only has to exchange addresses (and other business functions) and does not care what the other party wants to do with this information. As endpoints, they can choose P2P or proxy normally, and will not affect the underlying implementation of the relay server because of their own needs and functions. On TCP, port multiplexing is a good fit for this process. But in Quic, there is no way to adapt to this process, the key is that it cannot be reused.

However, it sounds like libp2p doesn't seem to be able to do my work, I'm using it to do the client-side p2p function, and the relay implementation shouldn't be tied to one of the features (unless that's all it does, but I have other businesses)

Jorropo commented 12 months ago

I don't understand what you are doing, do you have open source code you could show please ?

peakedshout commented 12 months ago

I don't understand what you are doing, do you have open source code you could show please ?

I'm sorry, but at the moment my project has a bit of code, and I can't fully show it at the moment, but the workflow that the client wants to implement looks like this

  1. relay <--reg-- server
  2. client --want link server--> relay <--keep-- server
  3. client <--server info-- relay --client info--> server 4.1 [relay type] client <--proxy--> relay <--proxy--> server 4.2 [p2p type] client <--p2p--> server This is just one of many business features, and I don't think I can replace the rest of the listeners that use this listener in order to achieve this.
Jorropo commented 12 months ago

What is your relay protocol ? Can you try a custom relay v2 which has an allow-list for infinite resource for connections to and from the server ? https://pkg.go.dev/github.com/libp2p/go-libp2p@v0.32.1/p2p/protocol/circuitv2/relay

peakedshout commented 12 months ago

What is your relay protocol ? Can you try a custom relay v2 which has an allow-list for infinite resource for connections to and from the server ? https://pkg.go.dev/github.com/libp2p/go-libp2p@v0.32.1/p2p/protocol/circuitv2/relay

The underlying layer is just TCP or QUIC protocol, but it will do a lot of things on the connection to complete the custom protocol. If it's just the client by changing the host. Host is the most ideal. Can relay v2 do this? I think I could give it a try

peakedshout commented 12 months ago

What is your relay protocol ? Can you try a custom relay v2 which has an allow-list for infinite resource for connections to and from the server ? https://pkg.go.dev/github.com/libp2p/go-libp2p@v0.32.1/p2p/protocol/circuitv2/relay

I tried it and found it didn't work for me. I rethought whether it was possible to generate a native dialer from the host to generate a native connection. This way I can use this native connection to communicate with the relay server and exchange information.

marten-seemann commented 11 months ago

I can see how injecting a quic.Transport (as in quic-go transport) can be useful. It's also not clear how an API would look for this, since the listener would somehow need to demultiplex connections intended for libp2p and connections not intended for libp2p. It's also unclear how ReadNonQUICPacket would work, which will be needed for #2446.

peakedshout commented 11 months ago

Great, it doesn't seem like I'm the only one I'm curious about. For some reason, for P2P, UDP has a higher success rate than TCP. Quic is undoubtedly a successful UDP reliable stream. If it can be injected by way, then it will undoubtedly allow libp2p to deal with more environments.