Multiple hosts - Githubissues

asbjornenge commented 10 years ago

After thinking some more about multiple host support, I don't think there is an elegant DNS solution to the problem.

One idea I have is to allow listening on multiple docker hosts, e.g. supporting multiple -s parameters, and leaving the networking between the hosts to some other mechanism.

This would keep skydock simple, focused and clean.

A setup similar to http://goldmann.pl/blog/2014/01/21/connecting-docker-containers-on-multiple-hosts/ might work...? Or any other setup. The more flexible skydock would not make assumptions about it and simply get out of it's way.

Thoughts?

kshep commented 10 years ago

Was just chatting with @crosbymichael in IRC about this. He mentioned he'd like a container-based approach, but it certainly seems like there's an option that could take advantage of an Open vSwitch VLAN that one sets up on all the hosts in a cluster before launching anything.

goldmann commented 10 years ago

@asbjornenge I wrote another blog posts and there is another solution: http://goldmann.pl/blog/2014/01/30/assigning-ip-addresses-to-docker-containers-via-dhcp/

asbjornenge commented 10 years ago

@goldmann certainly interesting! I have been avoiding ip conflicts by providing a different subnet to the docker daemon for each host (-bip). But, unless I am mistaken, dhcp does not solve the issue with using skydock for multiple host setups...? It is however a nice network setup, and provided that skydock could talk to multiple docker daemons, it would probably work very nicely :+1:

jhulten commented 10 years ago

I was thinking about this and was wondering how possible it would be to have two levels, private and public. Everything that connects to the bridge would be local and everything that binds to the public ip would be public. This might require two skydns services, one for .local and one for mycorp.com. Then only clustering the mycorp.com instance.

Thoughts?

crosbymichael commented 10 years ago

How would you know what ports are bound?

Michael Crosby

On Feb 6, 2014, at 5:28 PM, Jeffrey Hulten notifications@github.com wrote:

I was thinking about this and was wondering how possible it would be to have two levels, private and public. Everything that connects to the bridge would be local and everything that binds to the public ip would be public. This might require two skydns services, one for .local and one for mycorp.com. Then only clustering the mycorp.com instance.

Thoughts?

— Reply to this email directly or view it on GitHub.

asbjornenge commented 10 years ago

So, I had another idea... It requires the introduction of a skydock scheduler (container).

Registering hosts

(this could be done in number of ways, but in the spirit of skydock, lets leverage dns)

Hosts are registered in a DNS SRV record _hosts.domain.com_ with your "external" DNS service. This can be done either via your host spawn script, manually or by enabling the host to self register.

Scheduler

The skydock scheduler would be responsible for querying _hosts.domain.com_ and feeding that information to skydock. The scheduler is also responsible for spawning all containers in the cluster, including skydns and skydock. Since the scheduler knows where skydns lives and it's IP, it can pass a -dns flag along to all other containers.

Requirements

Preconfigured network (all the hosts need to be able to talk to each other). Separate docker subnets on each host.

Crazy stuff

The scheduler could perhaps even be responsible for starting the docker daemon on each host, thereby removing the last bit of host state (subnet). Taking that even further; I wonder about any potential drawbacks of running docker within docker...? Could perhaps all the hosts run vanilla docker, and the scheduler just deploy a "docker daemon container" on each host (with subnet arguments etc.) and use those to run all the application containers...? I don't know, I'm not a doc(ker).

Thoughts?