docker-compose does not create user defined network when extends: key is used

[benileo]

Description of the issue

docker-compose is not creating the user defined network present in a file which another service inherits from. In my example I create a service base-service which is supposed to attach to a user defined network called my_net. When this base service is inherited from, docker-compose does not create the user-defined network.

Base file (docker-compose-base.yml)

version: '2.1'
networks:
  my_net:

services:
  base-service:
    networks:
      - my_net

Service file (docker-compose-main.yml):

version: '2.1'
services:
  test:
    image: ubuntu
    entrypoint: sleep 3600
    extends:
      file: docker-compose-base.yml
      service: base-service

Output of docker-compose version

docker-compose version 1.22.0, build f46880f

Output of docker-compose config (Make sure to add the relevant -f and other flags)

$ docker-compose -f docker-compose-main.yml config
services:
  test:
    entrypoint: sleep 3600
    image: ubuntu
    networks:
      my_net: null
version: '2.1'

Error message

$ docker-compose -f docker-compose-main.yml up -d
ERROR: Service "test" uses an undefined network "my_net"

I'm guessing my_net: null is most likely an indication of an issue/bug

[shin-]

Hi @benileo Even if user-defined, my_net needs to be declared in the networks section as an external resource.

[benileo]

Based on reading the documentation, I thought external: true was only used if your network was created already, for example, docker network create my_net. I would like docker-compose to create this network (my_net) for me so I don't need to specify any additional bootstrapping.

[shin-]

Sorry, I misunderstood what you meant by "user defined network". The answer's the same though, that network needs to be declared if you want Compose to create it - just don't set the external property.

[benileo]

It is declared, but it’s not being created. That’s why I submitted this issue. El El mié, dic. 19, 2018 a las 4:32 p. m., Joffrey F < notifications@github.com> escribió:

Sorry, I misunderstood what you meant by "user defined network". The answer's the same though, that network needs to be declared if you want Compose to create it - just don't set the external property.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/docker/compose/issues/6421#issuecomment-448807733, or mute the thread https://github.com/notifications/unsubscribe-auth/AE3JDpp_HkVIidzXJM9cTST23W0GisvMks5u6tq5gaJpZM4ZbJiU .

[shin-]

It's missing from docker-compose-main.yml, which is the file you explicitly reference from the command line ; The extends key tells Compose to update the service with configuration from another file, but it isn't intended to import anything else in the file. You'll either need to add my_net to docker-compose-main.yml, or add -f docker-compose-base.yml to your Compose invocation (the latter will create a dummy service base-service though).

[benileo]

I see. I was envisioning it more like a python import. Thanks for clarifying. El El mié, dic. 19, 2018 a las 6:00 p. m., Joffrey F < notifications@github.com> escribió:

It's missing from docker-compose-main.yml, which is the file you explicitly reference from the command line ; The extends key tells Compose to update the service with configuration from another file, but it isn't intended to import anything else in the file. You'll either need to add my_net to docker-compose-main.yml, or add -f docker-compose-base.yml to your Compose invocation (the latter will create a dummy service base-service though).

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/docker/compose/issues/6421#issuecomment-448830839, or mute the thread https://github.com/notifications/unsubscribe-auth/AE3JDvH4KzvH0VqkHAdkM-CJio2VrseEks5u6u89gaJpZM4ZbJiU .

[moracabanas]

In my transition from docker-compose to swarm I found the lack of .env variable substitution a weird issue. So I found you can actually use docker-compose config as a preprocessor and pipe the result .env substitution to the deployment. From your supposed docker-compose.yml service deployment definition you can do:

docker-compose config | docker stack deploy -c - <name>

note - is the stantard input

From this docker-compose.yml:

version: '3.3'

services:

  traefik:
    # Use the latest Traefik image
    image: traefik:v2.2
    ports:
      # Listen on port 80, default for HTTP, necessary to redirect to HTTPS
      - 80:80
      # Listen on port 443, default for HTTPS
      - 443:443
    deploy:
      placement:
        constraints:
          - node.labels.traefik-public.traefik-public-certificates == true
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.middlewares.admin-auth.basicauth.users=${HTTPD_AUTH}
        - traefik.http.middlewares.https-redirect.redirectscheme.scheme=https
        - traefik.http.middlewares.https-redirect.redirectscheme.permanent=true

        - traefik.http.routers.traefik-public-http.rule=Host(`${URL_TRAEFIK_DASHBOARD}`)
        - traefik.http.routers.traefik-public-http.entrypoints=http
        - traefik.http.routers.traefik-public-http.middlewares=https-redirect

        - traefik.http.routers.traefik-public-https.rule=Host(`${URL_TRAEFIK_DASHBOARD}`)
        - traefik.http.routers.traefik-public-https.entrypoints=https
        - traefik.http.routers.traefik-public-https.tls=true

        - traefik.http.routers.traefik-public-https.service=api@internal

        - traefik.http.routers.traefik-public-https.tls.certresolver=le

        - traefik.http.routers.traefik-public-https.middlewares=admin-auth

        - traefik.http.services.traefik-public.loadbalancer.server.port=8080
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - traefik-public-certificates:/certificates

    command:
      - --providers.docker
      - --providers.docker.constraints=Label(`traefik.constraint-label`, `traefik-public`)
      - --providers.docker.exposedbydefault=false
      - --providers.docker.swarmmode
      - --entrypoints.http.address=:80
      - --entrypoints.https.address=:443
      - --certificatesresolvers.le.acme.email=${ACME_EMAIL}
      - --certificatesresolvers.le.acme.storage=/certificates/acme.json
      - --certificatesresolvers.le.acme.tlschallenge=true
      - --accesslog
      - --log
      - --api
    networks:
      - traefik-public

volumes:
    traefik-public-certificates:

networks:
  traefik-public:
    external: true

So running docker-compose config | docker stack deploy -c - <name> output the next error:

networks.traefik-public Additional property name is not allowed

Trying to debug this behaviour I run docker-compose config > docker-compose.parsed.yml and I got everything ok from the .env file but network definition:

----good parsed config----
...
networks:
      traefik-public: null
...
----good parsed config----

So my bets about this issue:

• There is a bug parsing docker-compose files with externally defined networks
• You are not supposed to use the same .env workflow from docker compose in anyways to run Swarm stacks because of this issues/behaviour.

How can I fix this? My entire workflow is based on docker-compose.yml variable definitions and .env substitution I do not know a better aproach for the scale I am working on and this erratic behaviour is hard to understand from the point I am trying to reuse the same tools docker provides but in a harder layer.