weaveworks / weave

Simple, resilient multi-host containers networking and more.
https://www.weave.works
Apache License 2.0
6.61k stars 667 forks source link

cope with differences in control and data plane topology #374

Open rade opened 9 years ago

rade commented 9 years ago

At the moment (modulo #372 and #373), the tcp control plane topology and udp data plane topology are synchronised, except for transitory conditions; we can only have udp connectivity between peers if we have tcp connectivity and vice versa.

That is unnecessarily limiting. In particular, it rules out deployments where udp connectivity is somewhat more broken than tcp connectivity. This will become even more of an issue once we consider alternative data path implementations.

rade commented 9 years ago

IMO it is reasonable to limit ourself to data plane topology being a sub-graph of control plane topology. I cannot think of a realistic scenario where that would be inadequate.

In which case our current representation of topology is in fact sufficient; the 'established' flag on a connection indicates presence of udp connectivity.

Things that are missing / would need changing:

rade commented 8 years ago

I think all the above actually has been done. What we haven't done, to the best of my knowledge, is test the main proposition of this issue. @awh

rade commented 8 years ago

we need a different mechanism to deal with asymmetric udp connectivity than tearing down the entire connection (including tcp)

We are not doing that yet - the TCP connection is torn down when dataplane heartbeating fails.

bboreham commented 6 years ago

For really large clusters it would be interesting to have a not-fully-connected control plane (say a hypercube) but form data-plane tunnels between arbitrary peers when they actually talk to each other.