gliderlabs / docker-consul

Dockerized Consul
MIT License
1.07k stars 286 forks source link

Question: Consul on clustered docker environment #98

Closed djenriquez closed 7 years ago

djenriquez commented 9 years ago

Hello,

I'm fairly new to Consul (2nd day of POC) and hoping someone could shed light on how Consul would look like in a clustered docker environment. Would Consul need to be installed on every node in the cluster? Would there be a Consul leader that services would look to for connection settings? With the use of a service discovery registering tool like Registrator, could we instead distribute Registrator to the nodes to then point to Consul?

Thank you in advance.

shibli786 commented 7 years ago

In the production environment you must have 3 consul server to form a cluster. These consul server containers can be schedule on any node. In the cluster running Swarm there is no mechanism to know how containers will be schedule on the different node. How ever docker swarm internally uses LibKV for service discovery to store these information but these information are not available to user. So we must use external Service discovery tool like consul,etcd,zookeeper ect. These tool are used as service registry by the service registrator. when ever new container is created Registrator inspect that container using linux socket (docker.sock) and register the information like ip address , port etc in the service registry and these information are replicated on each consul server by the leader. We have cluster of node forming swarm cluster and inside swarm cluster we choose three or five node forming another cluster for the consul server. The outer Swarm cluster and inner consul cluster both uses the RAFT consensus algorithm to select the leader. According to RAFT consensus algorithm there must be odd number of nodes >=3 to form a cluster.

You should setup your Registrator on each node from where docker events command can be fired like run start , scale etc. 1) If Registrator needs to run on a separate machine rather than where the Consul container is running then we need to do the following //IP of node where consul server is running export CONSUL_IP=// ip of any node where consul server is running export HOSTNAME= //name of host name where registrator is running

docker run -d -v /var/run/docker.sock:/tmp/docker.sock --name registrator -h $HOSTNAME gliderlabs/registrator:latest consul://$CONSUL_IP:8500

2)If Registrator is on the same machine as where the Consul container is, then we need to do the following: docker run -d -v /var/run/docker.sock:/tmp/docker.sock --name registrator -h $HOSTNAME gliderlabs/registrator:latest consul://$HOST_IP:8500

in the both method we are mounting the docker.sock to the registrator container. It is basically a linux socker which is used by docker daemon for listening the eventd fired from the docker client (CLI) eg docker run, start ,stop etc.