Open thormme opened 3 months ago
This isn't the same issue as #11387 as here this is the docker engine reporting error: Error response from daemon: network swarm-overlay not found
Can you please confirm you can use docker run --network swarm-overlay ...
to run equivalent container on worked node with this swarm setup ?
I'm running into this exact same issue using Docker Compose 2.27.0. I can confirm that I can use docker run -it --name alpine1 --network test-net alpine
from the official documentation. I walked through the entirety of the "Use an overlay network for standalone containers" and it worked as expected.
However, using docker compose files, I also get the error Error response from daemon: network <my network name here> not found
message using docker compose up -d
.
I am having the exact same issue. Docker Compose version v2.27.1 @ndeloof docker run --network swarm-overlay works and compose doesn't
btw is the downgrade workaround needed for both leader and worker node?
@inql I have not tested this as our scripts set versions for all nodes.
Hey there, also affected by this bug.
If you don't want to downgrade another workaround is to create a container and attach it to the network. It then appears in the list and docker compose no longer complains
docker run -dit --name keep-alive --network --restart=always <network_name> alpine
Adding --restart=always
will ensure that it survives restarts of the docker daemon, etc.
My versions in case it is useful:
docker version
Client: Docker Engine - Community Version: 27.0.3 API version: 1.46 Go version: go1.21.11 Git commit: 7d4bcd8 Built: Sat Jun 29 00:02:50 2024 OS/Arch: linux/amd64 Context: default
Server: Docker Engine - Community Engine: Version: 27.0.3 API version: 1.46 (minimum version 1.24) Go version: go1.21.11 Git commit: 662f78c Built: Sat Jun 29 00:02:50 2024 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.7.18 GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e runc: Version: 1.7.18 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0
docker compose version
Docker Compose version v2.28.1
As in above, sorry did not realise that @michaelmcandrew also mentioned this but at least this comment confirms his findings: https://github.com/docker/compose/issues/11894#issuecomment-2206522846
I tested this issue and noticed that if there exists running container which has connection to the external overlay network (started with docker run ...
and visible in docker network ls
), then the compose is able to connect to the external overlay network.
So, without knowing anything about internals, the problem might have something to do with not checking for available external overlay networks but instead checking just internal networks (visible with docker network ls).
So as an additinal workaround it is possible to first start "dummy" container on workers via for example:
$ docker compose up -d
Error response from daemon: network <overlay-network> not found
$ run -dit --rm --name dummy-network-container --network <overlay-network> alpine
43924b1b25ac73373aac9120b55ac46fc1de3435ce26485682e11d6c06671936
$ docker compose up -d
[+] Running 1/0
✔ Container worker-service Started
$ _
I also checked downgrading and for Ubuntu 22.04 it worked, so I think I will be using downgraded version for now myself.
sudo apt-get remove docker-compose-plugin && sudo apt-get install docker-compose-plugin=2.21.0-1~ubuntu.22.04~jammy
$ docker version
Client: Docker Engine - Community
Version: 27.0.3
API version: 1.46
Go version: go1.21.11
Git commit: 7d4bcd8
Built: Sat Jun 29 00:02:33 2024
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 27.0.3
API version: 1.46 (minimum version 1.24)
Go version: go1.21.11
Git commit: 662f78c
Built: Sat Jun 29 00:02:33 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.18
GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc:
Version: 1.7.18
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
$ docker compose version
Docker Compose version v2.28.1
@kulpsin docker network ls
indeed does not detect overlay networks created on another swarm node (not sure about the reason, but that's what we get with the engine API) until it is used by some container. So Docker Compose can't check network existence, but should detect swarm is enabled and ignore error (assuming container create will fail if there's an actual missing network). See https://github.com/docker/compose/blob/11d5ecdc75ab96214f35db4cdc0361ee080d1c07/pkg/compose/create.go#L1334-L1340
Not sure why this doesn't work as expected, need to setup a test environment and try to reproduce this bug
With the original compose.yml it would generate swarm-netword-overlay_swarm-overlay
network
...and then the worker
would not be able to find the external network as expected
By adding the name: swarm-overlay
on the network it made it work for me for version v2.28.1
docker compose up -d
...
service:
image: service-image
container_name: service
networks:
- swarm-overlay
restart: unless-stopped
...
networks:
swarm-overlay:
name: swarm-overlay <----
attachable: true
driver: overlay
after this it generates the following result for docker network ls
and now the worker is referencing the right network
To flesh out my steps to reproduce a bit more, since they are slightly different from the ones mentioned above, I created a swarm network on the lead node with docker network create --driver overlay test --attachable
.
This network was not visible on the worker node (expected I think because nothing was connected).
However, I was not able to connect to it with the below networks section in a compose.yaml
on the worker node.
networks:
test:
external: true
I created the following container on the worker node docker run -dit --name keep-alive --network test --restart=always alpine
I was then able to connect using the above networks section in a compose.yaml
on the worker node.
Hope that help with the reproduction!
Description
Issue: Swarm worker hosts fail to attach to manager node overlay networks unless a container has been manually started and attached to the network using
docker run --network swarm-overlay
Expected Behavior: This should automatically attach to the overlay network and it should be visible in the docker network info.
Workaround: The only solution I have found is to downgrade to an earlier version (
2.21.0-1
) ofdocker-compose-plugin
I believe this is the same issue as https://github.com/docker/compose/issues/11387 but i couldn't find any open bugs with the same issue.
Thanks for any help with this!
Steps To Reproduce
I created a custom overlay network on the swarm manager node.
This correctly created the network and attached the relevant container to it.
I then joined a worker host to the swarm and attempted to connect a container to the overlay network.
docker compose up -d worker-service
This errors with:Compose Version
Docker Environment
Anything else?
No response