[Question] Router Reflector and Node Node Mesh comparision

[gyliu513]

This is related to https://github.com/projectcalico/calico/issues/1839

I found that the proposal for route reflector is that when the cluster size is >200 nodes, but if cluster size <=200, it is suggested to use node node mesh.

So here my question is does there are any side effect if I use router reflector when the cluster size <=200 nodes? Does there are any performance downgrade when I switch from node node mesh to router reflector?

/cc @caseydavenport any comments for this? Thanks!

[tmjd]

The benefits I'm aware of are less overhead when reaching larger cluster sizes and also I believe improved behavior when restarting calico-node (Bird restarts). Where are you finding that recommendation of >200? I thought it was suggested around 100 nodes to consider using route reflectors?

I don't think there are any downsides to using route reflectors aside from the obvious additional component and management of them.

[gyliu513]

Thanks @tmjd . Based on our test, with node to node mesh, calico node will fail to start if the cluster size reach 700, and we do have a cluster which have 500 nodes works fine with node to node mesh mode.

The number 200 actually got from twitter https://twitter.com/projectcalico/status/950543850925260800 ;-)

One more question, if there is no downgrade for RR, why not use RR as the default mode for BGP router management instead of node to node mesh?

[tmjd]

The only downside (I'm aware of) is that it is necessary to run and manage the route reflectors to use them. The majority of users, especially small clusters, will benefit from the simplification of using node-to-node mesh instead requiring a route reflector.

[gyliu513]

@tmjd thanks, same as what my thinking, the RR will introduce some maintain effort but node node mesh is more simple for small cluster management, thanks.

/cc @hchenxa