thaJeztah
commented
5 years ago I see you have both stacks connected to the same (proxynet) network. What I expect is happening is;
The db and word containers are connected to the network, and will be reachable by any container connected to that network.
Services (and their containers) will be accessible through a number of hostnames. For example, the db service of the WP1 stack will be accessible through the following hostnames;
db(service name within the stack)db.proxynet(with network name suffix)WP1_db(prefixed with stack-name)WP1_db.proxynet(prefixed with stack-name, and network name suffix)
And the db service for the second stack (WP2);
db(service name within the stack)db.proxynet(with network name suffix)WP2_db(prefixed with stack-name)WP2_db.proxynet(prefixed with stack-name, and network name suffix)
This will be a problem, because in both stacks, you have the word container configured to be on the proxynet network and to connect to the database, using db as hostname.
Because the database of both WP1 and WP2 is connected to the same network, db could be either one, and in fact, Docker's embedded DNS will use a "round robin" load balancing for those IPs (thus randomly return the IP for db (of the WP1 stack) and db (of the WP2 stack)).
Reproduction of the problem
Here is a simple stack to reproduce this (I'm deploying on a single-node swarm, so all containers will be on my local node so that I can easily docker exec into each container);
Create the "proxynet" network
docker network create -d=overlay --scope=swarm proxynetDeploy stack WP1 (I use "heredoc" so that I don't have to create a file first :sweat_smile:)
docker stack deploy -c- WP1 <<EOF
version: '3.5'
services:
db:
image: emilevauge/whoami
hostname: WP1-db
networks:
- proxynet
word:
image: nginx:alpine
networks:
- proxynet
networks:
proxynet:
name: proxynet
external: true
EOFDeploy stack WP2:
docker stack deploy -c- WP2 <<EOF
version: '3.5'
services:
db:
image: emilevauge/whoami
hostname: WP2-db
networks:
- proxynet
word:
image: nginx:alpine
networks:
- proxynet
networks:
proxynet:
name: proxynet
external: true
EOFWait for the services to be up;
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
de5f0ec74aa0 nginx:alpine "nginx -g 'daemon of…" 34 seconds ago Up 33 seconds 80/tcp WP2_word.1.r4f73rmyte0tx17wh157a9px1
bb4a03cea164 emilevauge/whoami:latest "/whoamI" 37 seconds ago Up 35 seconds 80/tcp WP2_db.1.9z4m1lknokoz9bw5unt2leaud
9683e95dd143 nginx:alpine "nginx -g 'daemon of…" 47 seconds ago Up 44 seconds 80/tcp WP1_word.1.ldr16g26n2k3kxlygdzsx6fzj
61320520dd4f emilevauge/whoami:latest "/whoamI" 49 seconds ago Up 44 seconds 80/tcp WP1_db.1.x65h40wyihg0ru8vmtra3b3wmNow open a shell in the container for WP1_word:
docker exec -it WP1_word.1.ldr16g26n2k3kxlygdzsx6fzj shInstall some tools (dig, curl) in the container;
apk add --no-cache bind-tools curlObserve that all the hosts I listed work; but also that IP-addresses for both "db" services are returned (I removed some of the dig output to make it a bit more readable);
/ # dig db
;; QUESTION SECTION:
;db. IN A
;; ANSWER SECTION:
db. 600 IN A 10.0.1.23
db. 600 IN A 10.0.1.19db.proxynet (so, with the network-name suffix)
/ # dig db.proxynet
;; QUESTION SECTION:
;db.proxynet. IN A
;; ANSWER SECTION:
db.proxynet. 600 IN A 10.0.1.23
db.proxynet. 600 IN A 10.0.1.19Using the WP1_ prefix will only return the db service for WP1
/ # dig WP1_db
;; QUESTION SECTION:
;WP1_db. IN A
;; ANSWER SECTION:
WP1_db. 600 IN A 10.0.1.19However, the WP2 db service is accessible, even though we're in a container of the word service for the WP1 stack;
/ # dig WP2_db
;; QUESTION SECTION:
;WP2_db. IN A
;; ANSWER SECTION:
WP2_db. 600 IN A 10.0.1.23So, what happens if I try to connect to db? Let's try a few times:
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.20
IP: 172.18.0.4
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*
/ # curl db
Hostname: WP2-db
IP: 127.0.0.1
IP: 10.0.1.24
IP: 172.18.0.6
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.20
IP: 172.18.0.4
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.20
IP: 172.18.0.4
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*I'm randomly connected to the db service for WP1 or WP2.
As a result, your WordPress service will randomly connect to one database or the other, and (given that WordPress stores the full URL in the database), will thus send a redirect if the database doesn't match the website.
How to resolve this?
You need to use multiple networks. Don't connect an internal service (in this case; a database) to a network that is shared with other stacks.
Connecting internal services to a shared network is a security risk; other stacks can now connect to your database (and those stacks may be managed by "someone else", and now have access to your database :scream:).
In your example, the db services are connected the proxynet network; from the name of that network, I suspect this network is used by a front-end proxy (e.g. nginx-proxy or https://traefik.io), which means that the database may even become publicly accessible.
Docker networks are a "sandbox" mechanism, and allow you to isolate services, only giving other services access if they are connected to the same network.
In this example, you want;
- the WP1 WordPress service to be able to connect to the WP1 database
- the WP2 WordPress service to be able to connect to the WP2 database
- Your proxy service to be able to connect to both the WP1 and WP2 WordPress services
- the WP1 WordPress service not being able to connect to the WP2 database (and vice-versa)
- the WP2 WordPress service not being able to connect to the WP1 database (and vice-versa)
- the proxy service not being able to connect to any of the database services.
So, to realize this;
- create a proxynet network, connected with the proxy service
- all services that should be accessible by the proxy service will be connected to that network (in this case the WP1 WordPress and WP2 WordPress services)
- each stack will have its own "private" network; this network will be used for services in the same stack to communicate. If needed, a stack can have multiple "private" networks; for example if the stack is more complex, and there are distinct groups of services that need to communicate, but need to be isolated from other services in the same stack.
- The database services will be connected to the private network
- The WordPress services will also be connected to the private network (those services are thus connected to two networks; the "private" network to connect to the database, and the "proxynet" network to connect with the proxy).
Updated example
Here's what the example looks like with those changes:
Create the "proxynet" network. This network is shared with the proxy server, and other stacks.
docker network create -d=overlay --scope=swarm proxynetJust to illustrate; the "proxy" stack, which publishes its ports, so will be publicly accessible:
docker stack deploy -c- proxystack <<EOF
version: '3.5'
services:
proxy:
image: nginx:alpine
networks:
- proxynet
networks:
proxynet:
name: proxynet
external: true
EOFDeploy stack WP1
docker stack deploy -c- WP1 <<EOF
version: '3.5'
services:
db:
image: emilevauge/whoami
hostname: WP1-db
networks:
- private
word:
image: nginx:alpine
networks:
- private
- proxynet
networks:
private:
proxynet:
name: proxynet
external: true
EOFDeploy stack WP2:
docker stack deploy -c- WP2 <<EOF
version: '3.5'
services:
db:
image: emilevauge/whoami
hostname: WP2-db
networks:
- private
word:
image: nginx:alpine
networks:
- private
- proxynet
networks:
private:
proxynet:
name: proxynet
external: true
EOFAfter the stacks have been deployed, you'll see that the three networks have been created proxynet, WP1_private and WP2_private;
docker network ls
NETWORK ID NAME DRIVER SCOPE
x87foyu2y37q WP1_private overlay swarm
jwsym6k7rr0u WP2_private overlay swarm
7042da47ef5f bridge bridge local
3e32a906b55f docker_gwbridge bridge local
ac25d483c79e host host local
9ebmyr0vctzj ingress overlay swarm
69771e2906b2 none null local
1598r4x4j4e4 proxynet overlay swarmNow, let's check connectivity again;
Open a shell in the container for WP1_word:
docker exec -it WP1_word.1.pwt0ukt8lz8v5ews5duwfeu9k shInstall some tools (dig, curl) in the container;
apk add --no-cache bind-tools curlGet the IP address for the db host; notice that there's now only 1 IP address returned
dig db
;; QUESTION SECTION:
;db. IN A
;; ANSWER SECTION:
db. 600 IN A 10.0.1.5The db service for this stack is connected to the "private" network, so can also be accessed with that suffix;
dig db.WP1_private
;; QUESTION SECTION:
;db.WP1_private. IN A
;; ANSWER SECTION:
db.WP1_private. 600 IN A 10.0.1.5The WordPress container is connected to the proxynet network, so is also able to connect to the WordPress container of the other stack;
curl WP2_word
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...As well as the proxy service:
curl proxystack_proxy
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...But cannot connect to the other stack's database;
curl WP2_db
curl: (6) Could not resolve host: WP2_dbAnd, as expected: connecting to the db service will always connect with the db service in the same stack;
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.6
IP: 172.18.0.5
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.6
IP: 172.18.0.5
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*
/ # curl db
Hostname: WP1-db
IP: 127.0.0.1
IP: 10.0.1.6
IP: 172.18.0.5
GET / HTTP/1.1
Host: db
User-Agent: curl/7.61.1
Accept: */*Success! :tada:
So, what's left?
In the above example, both the WP1 and WP2 stack are connected to the proxynet network. This does mean that both stacks can connect to the "WordPress" service from the other stack (WP1_word can connect to WP2_word, and vice-versa). This is likely "ok" (assuming the WordPress services will be accessible publicly anyway), however, if you don't want to allow this, and only want the proxy to be able to connect to both, you need to give each stack a "public" network, and connect the proxy to that network.
Something like this;
compose file for WP1 and WP2:
version: '3.5'
services:
db:
image: emilevauge/whoami
networks:
- private
word:
image: nginx:alpine
networks:
- private
- public
networks:
private:
public:compose file for the proxy; the proxy service is connected to the "public" networks of both the WP1 and WP2 stack:
version: '3.5'
services:
proxy:
image: nginx:alpine
networks:
- WP1_public
- WP2_public
raven-wing
Expected behavior
When I go to WP1_IP I get app on first WP container When I go to WP2_IP I get app on second WP container
Actual behavior
When I go to WP1_IP I get app on first wordpress container, but after refreshing 3 times I get contents of the second container. I get response 301 which is redirecting to first container. Same goes for WP2_IP addres.
Steps to reproduce the behavior
wordpress.yml
network.yml:
docker stack deploy --compose-file wordpress.yml WP1 docker stack deploy --compose-file wordpress.yml WP2
curl -I ${WP1_IP}
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.)
running on docker-machine version 0.13.0, build 9ba6da9