Open rade opened 9 years ago
IMO it is reasonable to limit ourself to data plane topology being a sub-graph of control plane topology. I cannot think of a realistic scenario where that would be inadequate.
In which case our current representation of topology is in fact sufficient; the 'established' flag on a connection indicates presence of udp connectivity.
Things that are missing / would need changing:
I think all the above actually has been done. What we haven't done, to the best of my knowledge, is test the main proposition of this issue. @awh
we need a different mechanism to deal with asymmetric udp connectivity than tearing down the entire connection (including tcp)
We are not doing that yet - the TCP connection is torn down when dataplane heartbeating fails.
For really large clusters it would be interesting to have a not-fully-connected control plane (say a hypercube) but form data-plane tunnels between arbitrary peers when they actually talk to each other.
At the moment (modulo #372 and #373), the tcp control plane topology and udp data plane topology are synchronised, except for transitory conditions; we can only have udp connectivity between peers if we have tcp connectivity and vice versa.
That is unnecessarily limiting. In particular, it rules out deployments where udp connectivity is somewhat more broken than tcp connectivity. This will become even more of an issue once we consider alternative data path implementations.