Combines some tooling for creating a good Docker Swarm Cluster.
Install Ubuntu on all VMs you're mean't to use in your Swarm Cluster
Install the latest Docker package on all VMs (https://docs.docker.com/engine/install/ubuntu/)
The best practise is to have
Use Caddy to handle TLS (with Let's Encrypt) and load balancing
Use a cloud LB to handle front TLS certificates and load balancing
caddy-server:
deploy:
labels:
- caddy.auto_https=off
- caddy_controlled_server=
...yourservice:
...
deploy:
placement:
constraints:
- node.role != managerOn one of the VMs:
docker swarm init on the first VM with role manager--advertise-addr to indicate which network to use for swarm communicationsdocker swarm token-info manager and keep to run on manager machinesOn machines selected to be managers (min 3)
--advertise-addr [localip] with a local IP that connects those machines if they are local so that you don't use a public IP for that (by using Internet link)docker swarm join --advertise-addr 10.120.0.5 --token ...On machines selected to be workers
docker swarm token-info worker and add --advertise-addr [localip] with a local IP that connects those machines if they are local so that you don't use a public IP for that (by using Internet link)docker swarm join --advertise-addr 10.120.0.5 --token ...Make Docker daemon configurations on all machines
echo '{"log-driver": "journald", "metrics-addr" : "172.18.0.1:9323", "experimental" : true, "default-ulimits": { "memlock": { "Name": "memlock", "Hard": -1, "Soft": -1 }, "stack": { "Name": "stack", "Hard": -1, "Soft": -1 }} }' > /etc/docker/daemon.json
service docker restartStart basic cluster services
git clone https://github.com/flaviostutz/docker-swarm-cluster.gitcreate.shOn one of the VMs, run curl -kLv --user whoami:whoami123 localhost and verify if the request was successful
If you need elasticity (need to grow or shrink server size depending on app traffic) a good topology would be to have some two cluster "sizes". One that we call "idle" that has the minimal sizing when few users are on, and a "hot" configuration when traffic is high.
For the "idle" state, we use:
For the "hot" state, we use:
...
placement:
preferences:
- spread: node.role
...Services will be accessible by URLs: http://portainer.mycluster.org http://dashboard.mycluster.org http://grafana.mycluster.org http://unsee.mycluster.org http://alertmanager.mycluster.org http://prometheus.mycluster.org
Services which don't have embedded user name protection will use Caddy's basic auth. Change password accordingly. Defaults to admin/admin123admin123
The following services will have published ports on hosts so that you can use swarm network mesh to access admin service directly when Caddy is not accessible
So point your browser to any public IP of a member VM to this port and access the service
# docker service ls -q > dkr_svcs && for i in `cat dkr_svcs`; do docker service update "$i" --detach=false --force ; done
for service in $(docker service ls -q); do docker service update --force $service; doneWARNING: User service disruption will happen while doing this as some containers will be stopped during this operation
docker swarm join-token worker to get a Swarm join token--advertise-addr [local-network-interface-ip] to the command if your host has multiple NICsdocker swarm join --token aaaaaaaaaaaa 10.120.0.2:2377 --advertise-addr 10.120.0.1PLACE IMAGE HERE
- caddy.tls=internal label to your service.create.sh for updating modified servicesexport $(cat .env) && docker stack... so that those parameters workexport $(cat .env) && docker stack deploy --compose-file docker-compose-ingress.yml ingressexport $(cat .env) && docker stack deploy --compose-file docker-compose-admin.yml adminexport $(cat .env) && docker stack deploy --compose-file docker-compose-metrics.yml metricsexport $(cat .env) && docker stack deploy --compose-file docker-compose-devtools.yml devtoolsFor HTTPS certificates, use Let's Encrypt in Load Balancers if you are using a first level domain (something like stutz.com.br). We couldn't manage to make it work with subdomains (like poc.stutz.com.br).
For subdomains, use certbot and create a wildcard certificate (ex.: *.poc.stutz.com.br) manually and then upload it to Digital Ocean's Load Balancer.
apt-get install letsencrypt
certbot certonly --manual --preferred-challenges=dns --email=me@me.com --server https://acme-v02.api.letsencrypt.org/directory --agree-tos -d *.poc.me.com