search

weaveworks / weave

Simple, resilient multi-host containers networking and more.
https://www.weave.works
Apache License 2.0
6.62k stars 670 forks source link

Allow filtering of discovered peers before connecting to them #2476

Open theartoflogic opened 8 years ago

theartoflogic commented 8 years ago

I recently ran into the max peer connection limit, which currently defaults to 30. After thinking about this, I thought it would be useful to have the ability to filter the list of auto-discovered peers so I can continue to use auto-discovery while ensuring I only directly connect to peers that matter to a given Weave host.

For instance, I have a set of API hosts that need to be able to connect to other hosts running Mongo or Redis. However, there are multiple Mongo and Redis hosts (they're running in a cluster), and the hosts could change. I also have a gateway server that I use as my default <peer> when launching a new host, using weave launch <peer>. That way I only ever need to hard-code the gateway peer when launching Weave and the new host will then connect to the other peers using auto-discovery.

However, it really only makes sense for the API servers to directly connect to the Mongo and Redis servers. I have other servers that don't need to talk to the API, but do need to talk to Mongo or Redis, so they need to be in the same Weave network. Using auto-discovery it would attempt to connect to all peers.

I suggest implementing a glob or regex pattern system to filter out auto-discovered peers before making connections to them. Imagine the following scenario:

  1. I launch a new API server (api-01) using weave launch --filter-peers "mongo-* redis-*" gateway (where gateway is the hostname to a server that has peer connections to every other host).
  2. Using auto-discovery, the gateway peer sends back the full list of peers to api-01.
  3. Before trying to connect to any peers, api-01 first filters the list of auto-discovered peers based on the pattern defined in the --filter-peers argument. That way it'll only connect to those peers with a hostname that starts with mongo- or redis-.
  4. If I add a new Mongo host to the Weave network (mongo-03), the gateway server will connect to it and then tell all other peers about the new host.
  5. api-01 will see a new peer was added, but before trying to connect to it it'll do the same filtering to see if it matches the glob pattern...which in this case it would.
  6. If I add another host other, it will be filtered out on api-01 and won't create a peer connection between api-01 and other.

This way I can limit the number of peer connections that any given host has, while ensuring that each host still connects to relevant peers on the Weave network to reduce the latency of multi-hop routing.

rade commented 8 years ago

Is the latency of multi-hop routing really that much of an issue for you? After all, 30 connections per node allows the construction of 2-hop topologies of close to 900 nodes.

theartoflogic commented 8 years ago

I've ended up writing a simple service that accomplishes my goals outlined here using what I'm calling a "weave connector" container, so feel free to implement this suggestion or not. I still believe this is something that would be beneficial for reasons beyond just latency (for example, less network traffic, less open sockets, which means less memory and CPU, etc.).

bboreham commented 7 years ago

some related conversation at #1621