weaveworks / weave

Simple, resilient multi-host containers networking and more.
https://www.weave.works
Apache License 2.0
6.62k stars 670 forks source link

Allow user to specify a subnet range for each docker host #1742

Open Crazee opened 8 years ago

Crazee commented 8 years ago

IPAM seems way too fussy and goes on strike if it has any small disagreement with you.

I can generate a unique subnet for all the containers on a host and can pass it in on "weave launch-router".

I'd like weave-router to handle the allocation of container addresses from this subnet as well as the routing between hosts. But do without the waiting for "peer consensus" part 'cuz the subnets I pick will never conflict.

I suspect that's not hard for a lot of folks who already have a database of the hosts they are spinning up.

But I don't want to allocate individual container addresses and can't add arguments to the "docker run" commands.

bboreham commented 8 years ago

Hi, this is an interesting idea, and probably can be done quite simply.

First, however, the word "subnet" implies routing; unless you set up a bunch of routes between them, a set of unique subnets would be unable to communicate. So I'm going to assume you meant "I can generate a unique sub-range of the address space for all the containers on a host", and you would put all your containers everywhere on the same subnet.

We already have --ipalloc-range to specify the range but, the current code will blow up when it sees you gave different ranges to each host. So to implement your suggestion would merely require an option to disable communication of IPAM data so they never find out. The cluster size on each host defaults to 1, so they will all exist quite happily in a clique of 1.

However, note also #1743 which would reduce the scope for IPAM to get into an argument.

Crazee commented 8 years ago

In my case I was going to allocate a /10 or /12 for weave as a whole and carve out a /23 or /24 for each host.

Another advantage of this is that docker is functional right away on each host and you don't get blocked with the bar closed until enough guests arrive at the party.

I actually tried --ipalloc-range with --ipalloc-default-subnet hoping to effect this but just got "connection shutting down due to error: IP allocation was seeded by different peers" for my effort.

All the tinkering with IPAM seems like propping up a fragile system that is guaranteed to break at some point when the right conditions manifest themselves. And it always seems to take down the whole of docker when it objects to something.

bboreham commented 8 years ago

As noted by @awh, you can now achieve what you asked for using the weave plugin:

Host1:

$ weave launch
$ docker network create --driver=weavemesh --subnet=10.3.0.0/16 --ip-range=10.3.0.0/24 mynet
$ docker run --net=mynet [my container args]

Host2:

$ weave launch
$ docker network create --driver=weavemesh --subnet=10.3.0.0/16 --ip-range=10.3.1.0/24 mynet
$ docker run --net=mynet [my container args]

etc. Put all the addresses in the same subnet, but with each host using a different sub-range within that.