libp2p / go-libp2p-pubsub

The PubSub implementation for go-libp2p
https://github.com/libp2p/specs/tree/master/pubsub
Other
309 stars 178 forks source link

Pubsub does not seem to work over RelayTransports #519

Closed steamraven closed 1 year ago

steamraven commented 1 year ago

I am having trouble getting pubsub messages to route when the peers are connected over Circuit Relay From my understanding, this is not using PubSub relay functionality, but using libp2p Circuit Relay transports to make a direct connection between the two peers

When I connect two peers directly, the pubsub messages seem to be routed correctly. But as soon as I force the connection to go over a Circuit Relay, messages stop flowing.

Here is a simple demo

https://gist.github.com/steamraven/41ec1867b7677d380555cc541c2bd2e7

The one thing I noticed in the PubSub tracing. When connected directly, I see a SendRPC to the other peer:

{"type":7,"peerID":"ACQIARIgmpXRAla42PScOFiGWQZG+epkAREMSCfurIAUNl91FSU=","timestamp":1676308810306010325,"sendRPC":{"sendTo":"ACQIARIgblqzIAP4VYmIcaZsMDhXM+7h/DE8AEWqn8NqFG1OE04=","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":9,"peerID":"ACQIARIgmpXRAla42PScOFiGWQZG+epkAREMSCfurIAUNl91FSU=","timestamp":1676308810306019028,"join":{"topic":"beetjuice"}}

But when connecting over Circuit Relay, the trace shows SendRPCs to the intermediate peers but not to the final connected peer

{"type":7,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803353654,"sendRPC":{"sendTo":"EiDTWJPkgrSNgwxlNBXmFQYSgZgiJgWdj8tlrdF4y7mQvw==","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":7,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803356773,"sendRPC":{"sendTo":"EiCwSlfUDsoTiAnxOadrEgRDM8N0A5HJvxzp2OIaeSEL/Q==","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":7,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803357439,"sendRPC":{"sendTo":"EiAbnqSfks+TC8JKwRg+eO0TWKHincaQMSacH9fVP/Zf0Q==","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":7,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803358163,"sendRPC":{"sendTo":"EiAGs2CKoAAnQEnrKK2OeTom/2+rKBp9O9d80Y63Rd+quw==","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":7,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803359032,"sendRPC":{"sendTo":"EiDBD5MZ2sNcJwprdM1kTLOs/B9u/IyCH46yglmf0YFPZA==","meta":{"subscription":[{"subscribe":true,"topic":"beetjuice"}]}}}
{"type":9,"peerID":"ACQIARIgyzqjnb52c/6BqVsSS7fDbVkxfEQLp7NkxPpcBER+Ygo=","timestamp":1676307932803365384,"join":{"topic":"beetjuice"}}

Am I doing something wrong?

vyzo commented 1 year ago

Are they transient v2 connections? Pubsub ignores such conns.

steamraven commented 1 year ago

@vyzo I am not sure about the transient part. I am connecting the peers using thier .../p2p-circuit mulltiaddresses. I am using all defaults, so I believe it is connecting with Circuit Relay v2. Since this is a full connection, I would assume that it is not transient, but I don't have enough experience with libp2p to know

vyzo commented 1 year ago

Unless you relay is explicitly specifying unlimited, ie if there is any limit, this is transient then. Pubsub is not interested in transient connections at all.

steamraven commented 1 year ago

@vyzo Thank you. The keyword transient got me pointed in the right direction. It appears V2 is only used for lightweight protocols like identity, ping, and hole-punching. And not for heavy protocols like pubsub. I see I need to enable the extra step of hole-punching to create a direct connection.

I wish the documentation on the circuit relay V2 and pubsub was a little more explicit on thos.

Thank you so much for your help