libp2p / go-libp2p-pubsub

The PubSub implementation for go-libp2p
https://github.com/libp2p/specs/tree/master/pubsub
Other
319 stars 185 forks source link

Connecting to unreachable peer breaks publishing for all #418

Open elgohr opened 3 years ago

elgohr commented 3 years ago

Scenario Network

[ Peer A ] - [ Peer B ] - [ Peer C ]

Other connections (let's say Peer C to Peer A) are just dropping packets.

Issue

Let's say Peer A connects to Peer B and Peer C connects to Peer B; then all peers get the message when Peer C is publishing via Gossip.

In the case that Peer C tries to connect to Peer A (and fails), then nobody gets messages. The error shown in the logs is https://github.com/multiformats/go-multistream/blob/master/client.go#L15 (via https://github.com/libp2p/go-libp2p/blob/master/p2p/host/basic/basic_host.go#L647) for Peer A

Hacky Workaround

Vendoring the dependency and setting the error in https://github.com/multiformats/go-multistream/blob/ff9dcd5c891b3ba194062358d52623047d85192f/client.go#L74 to something else results in a lot of peer declared dead but still connected; respawning writer-messages (from here https://github.com/libp2p/go-libp2p-pubsub/blob/master/pubsub.go#L529), but finally reestablishes the connection to all peers - so that all get the published message.

vyzo commented 3 years ago

this is not a bug in pubsub, it does not own the connections.

elgohr commented 3 years ago

I'm not sure about that. To my point of view pubsub also holds some connection-state https://github.com/libp2p/go-libp2p-pubsub/blob/cbb7bfc1f182e0b765d2856f6a0ea73e34d93602/gossipsub.go#L345 In this way, simply resetting the connection (https://github.com/libp2p/go-libp2p-examples/blob/master/relay/main.go#L77) doesn't work - as the peer still remains in pubsub backoff.

elgohr commented 3 years ago

I'm still trying to find a way to use

https://github.com/libp2p/go-libp2p-pubsub/blob/cbb7bfc1f182e0b765d2856f6a0ea73e34d93602/pubsub.go#L167

as the router is private

https://github.com/libp2p/go-libp2p-pubsub/blob/master/pubsub.go#L50

and it looks like there's no way to use

https://github.com/libp2p/go-libp2p-pubsub/blob/cbb7bfc1f182e0b765d2856f6a0ea73e34d93602/pubsub.go#L221

as there's no way to initialize a GossipSubRouter from outside the package

https://github.com/libp2p/go-libp2p-pubsub/blob/cbb7bfc1f182e0b765d2856f6a0ea73e34d93602/gossipsub.go#L168

also because of private variables