Network priority ignored for external networks

[pytz]

Not sure if this is intended behaviour or a bug.

I have a docker-compose file for an OpenVPN container which uses two external networks:

networks:
  astnet:
    external: true
    name: astnet
  ovpnnet:
    external: true
    name: ovpnnet
services:
  openvpn:
    ... left out intentionally ...
    networks:
      astnet:
        ipv4_address: 192.168.250.250
      ovpnnet:
        ipv4_address: 192.168.252.2
    ports:
    - 192.168.253.2:1194:1194/udp
    - 192.168.253.2:1195:1194/udp
version: '2.4'

The started container uses astnet for the default route:

bash-4.3# ip route
default via 192.168.250.1 dev eth0 
10.0.2.0/24 dev tun0 proto kernel scope link src 10.0.2.1 
192.168.3.0/24 via 10.0.2.2 dev tun0 
192.168.250.0/24 dev eth0 proto kernel scope link src 192.168.250.250 
192.168.252.0/24 dev eth1 proto kernel scope link src 192.168.252.2

and also in the matching iptables rule:

Chain DOCKER (24 references)
target     prot opt source               destination      
ACCEPT     udp  --  anywhere             192.168.250.250      udp dpt:openvpn

Now I would like to configure docker to use the ovpnet as default network. According to https://docs.docker.com/compose/compose-file/compose-file-v2/#priority and Issue #4645 it should be possible by setting a priority like this:

      astnet:
        ipv4_address: '192.168.250.250'
        priority: 200
      ovpnnet:
        ipv4_address: '192.168.252.2'
        priority: 300

But this doesn't change anything. astnet is still eth0 and has the default route. I also tried switching the priorities and changing the order of the networks but no success.

Version informations

docker version:

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:21:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

docker-compose version:

docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

[pytz]

Here a minimal working example.

First create both networks:

docker network create -d bridge --subnet=192.168.101.0/24 a_net
docker network create -d bridge --subnet=192.168.102.0/24 b_net

Start the following using docker-compose up -d:

networks:
  a_net:
    external: true
    name: a_net
  b_net:
    external: true
    name: b_net
services:
  my_service:
    image: alpine:3.7
    container_name: my_service
    networks:
      a_net:
        ipv4_address: 192.168.101.2
        priority: 100
      b_net:
        ipv4_address: 192.168.102.2
        priority: 200
    command: sleep 3600
    ports:
    - 127.0.0.1:1194:1194/udp
    - 127.0.0.1:1195:1194/udp
version: '2.4'

Check the output of docker exec -it my_service ip route:

default via 192.168.101.1 dev eth0 
192.168.101.0/24 dev eth0 scope link  src 192.168.101.2 
192.168.102.0/24 dev eth1 scope link  src 192.168.102.2

[jcsirot]

Hello @pytz thank you for filling this issue

[pcellix]

Hello I would like to work on this issue I'm new so I would new info how can I be assigned

[pcellix]

Is it really a bug as it does state regarding routing metric but the priority of connecting

[pytz]

I'm not sure if it is a bug. You are right as the docker-compose reference (https://docs.docker.com/compose/compose-file/compose-file-v2/#priority) just says it indicates the order in which networks should be connected. But I still don't understand which interface becomes the "default" route. My guess would be the first interface (= eth0). So changing the order in which the interfaces get connected to the docker container should influence the eth0, eth1, ... assignment.

But going back to the minimal working example above the priority doesn't seem to matter (note the different priority):

networks:
  a_net:
    external: true
    name: a_net
  b_net:
    external: true
    name: b_net
services:
  my_service:
    image: alpine:3.7
    container_name: my_service
    networks:
      a_net:
        ipv4_address: 192.168.101.2
        priority: 100
      b_net:
        ipv4_address: 192.168.102.2
        priority: 200
    command: sleep 3600
    ports:
    - 127.0.0.1:1194:1194/udp
    - 127.0.0.1:1195:1194/udp
version: '2.4'

# docker exec -it my_service ip route
default via 192.168.101.1 dev eth0 
192.168.101.0/24 dev eth0 scope link  src 192.168.101.2 
192.168.102.0/24 dev eth1 scope link  src 192.168.102.2

Changing the priority has no result:

networks:
  a_net:
    external: true
    name: a_net
  b_net:
    external: true
    name: b_net
services:
  my_service:
    image: alpine:3.7
    container_name: my_service
    networks:
      a_net:
        ipv4_address: 192.168.101.2
        priority: 200
      b_net:
        ipv4_address: 192.168.102.2
        priority: 100
    command: sleep 3600
    ports:
    - 127.0.0.1:1194:1194/udp
    - 127.0.0.1:1195:1194/udp
version: '2.4'

# docker exec -it my_service ip route
default via 192.168.101.1 dev eth0 
192.168.101.0/24 dev eth0 scope link  src 192.168.101.2 
192.168.102.0/24 dev eth1 scope link  src 192.168.102.2

[karthanistyr]

Has this bug been acknowledged? I repeatedly get the wrong interface (network) assigned to the default route in my containers.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[karthanistyr]

This is still not addressed (or even acknowledged) @shin- any input?

[stale[bot]]

This issue has been automatically marked as not stale anymore due to the recent activity.

[jfellus]

It seems to be the case for interal networks as well, see https://gist.github.com/jfellus/cfee9efc1e8e1baf9d15314f16a46eca to reproduce. Lexical order of network names seems to always superseed any priorities you set. To be precise, this has an impact on the order in which your container gets connected to networks, but it is then ignored by docker that keep on allocating ethX based on lexical order of network names

[tevino]

@jfellus Thanks for the lexical order workaround!

[newsages]

lexicon ,, tabs ,,, typical of python people ... ;)

[corviv]

@jfellus if you use IPv6 (enable ipv6: true), in this case the lexical order stops working - a network that uses IPv6 becomes more priority

tested at v3.9: 2 networks, one of them uses IPv6

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it had not recent activity during the stale period.

[jpbberry]

Still having this issue, quite annoying how it's just being ignored

[iglesiasroberto14]

Same problem here. Using lexic order workaround didn't work.

Lexic order applies to the real name of the network, not the "alias".

As an example:

networks:
  a_net:
    external: true
    name: zz_net
  b_net:
    external: true
    name: aa_net
services:
  my_service:
    image: alpine:3.7
    container_name: my_service
    networks:
      a_net:
        ipv4_address: 192.168.101.2
        priority: 200
      b_net:
        ipv4_address: 192.168.102.2
        priority: 100
    command: sleep 3600
    ports:
    - 127.0.0.1:1194:1194/udp
    - 127.0.0.1:1195:1194/udp

This would result in 192.168.101.2 being applied to eth1 as real network name is zz_net and is lexically greater than real name of b_net, aa_net.

[amenk]

Seems to be similar / duplicate of https://github.com/docker/compose/issues/8561 ?