Networks created by docker-compose conflict with private ips used @dcs

[lloyd-uot-cs]

We configured the docker daemon to have the docker0 interface use 192.168.152.0 instead of the default 172.17.0.0 (which collides with red.sandbox on dcs machines).

This works when just the docker daemon is running:

tapp@doc:~/tapp$ netstat -ar Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default router.pub.sand 0.0.0.0 UG 0 0 0 ens160 localnet 255.255.255.0 U 0 0 0 ens160 192.168.152.0 255.255.255.0 U 0 0 0 docker0

Once we launch the tapp app, docker-compose spins up two more network interfaces, (tapp_internal and tapp_external) and they are allocated 172.17.0.0 and 172.18.0.0, which again collide:

tapp@doc:\~/tapp$ docker-compose up -d Creating network "tapp_internal" with driver "bridge" Creating network "tapp_external" with driver "bridge" Creating tapp_postgres_1 Creating tapp_rails-app

tapp@doc:~/tapp$ netstat -ar Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default router.pub.sand 0.0.0.0 UG 0 0 0 ens160 *172.17.0.0 255.255.0.0 U 0 0 0 br-c489d67e84d7* 172.18.0.0 255.255.0.0 U 0 0 0 br-3824b523e9aa localnet 255.255.255.0 U 0 0 0 ens160 192.168.152.0 255.255.255.0 U 0 0 0 docker0

Unlike docker0, this isn't a docker install issue, it is specific to tapp because if I bring up a generic docker container that serves web pages it doesn't add additional interfaces and I can access the host from a red net machine.

pocadmin@doc:\~/mytest$ docker run -it --rm --name my-apache-app -p80:80 -v "$(pwd)":/usr/local/apache2/htdocs/ httpd:2.4

tapp@doc:~/tapp$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ae8beb922348 httpd:2.4 "httpd-foreground" 3 seconds ago Up 3 seconds 0.0.0.0:80->80/tcp my-apache-app tapp@doc:\~/tapp$ netstat -ar Kernel IP routing table Destination Gateway Genmask Flags MSS Window irtt Iface default router.pub.sand 0.0.0.0 UG 0 0 0 ens160 localnet 255.255.255.0 U 0 0 0 ens160 192.168.152.0 255.255.255.0 U 0 0 0 docker0

I assume this explains something about a solution https://docs.docker.com/compose/compose-file/#external-1

external

If set to true, specifies that this network has been created outside of Compose. docker-compose up will not attempt to create it, and will raise an error if it doesn’t exist.

lloyd

[jmzaleski]

I have an idea (Peter Marbach made me think this). Clue: we are trying to route non-routable IPs Hence: don't

Instead, put the reverse proxy onto a separate VM -- hence it's routing won't be polluted by all of dockers fancy bridge network adapters.

Like so:

[lloyd-uot-cs]

Fixed on doc.pub.sandbox with:

# docker network create --driver=bridge --subnet=192.168.153.0/24 --ip-range=192.168.153.0/24 --gateway=192.168.153.254 frontend # docker network create --driver=bridge --subnet=192.168.154.0/24 --ip-range=192.168.154.0/24 --gateway=192.168.154.254 backend

Changed \~tapp/tapp/docker-compose.yml to have :

networks: # Names of the networks will be prefixed with project name by docker backend: external: true

frontend: external: true

Rebuilt app, populated database from backup, now can reach http://doc.pub.sandbox:3000/index.html/summary from a red subnet machine.

I have a /etc/daemon/daemon.json file that I think does the docker network create at boot time... still testing.

[lloyd-uot-cs]

I don't have the daemon.json file syntax right, but will fiddle with the network(s) to remove their ability to route for