[Bug]: Deployments based on Docker Compose cannot join the coolify network and cannot connect to preset services, such as MySQL.

[Huangwh826]

Description

Deployments based on Docker Compose can only communicate within their own network, even though I declared networks in the yml.

# container networks
root@racknerd-d8d60a:/data/coolify/# docker ps -q | xargs docker inspect -f '{{ .Name }} {{ .NetworkSettings.Networks }}'
/halo-asosswg map[asosswg:0xc00016e460]  # halo
/a4gs0k0 map[coolify:0xc00016e540]  # mysql
/coolify-proxy map[asosswg:0xc0004e4000 coolify:0xc0004e40e0 dss44sc:0xc0004e41c0 n4wokcs:0xc0004e42a0]
/coolify map[coolify:0xc00016e000]
/coolify-redis map[coolify:0xc0004e4380]
/coolify-realtime map[coolify:0xc00016e0e0]
/coolify-db map[coolify:0xc00016e460]

Minimal Reproduction (if possible, example repository)

version: v4.0.0-beta.239

docker-compose

version: '3'
services:
  halo:
    image: 'halohub/halo:2.13'
    container_name: halo
    restart: 'on-failure:3'
    volumes:
      - './halo2:/root/.halo2'
    ports:
      - '9001:8090'
    command:
      - '--spring.r2dbc.url=r2dbc:pool:mysql://a4gs0k0:3306/halo'
      - '--spring.r2dbc.username=****'
      - '--spring.r2dbc.password=*****'
      - '--spring.sql.init.platform=mysql'
      - '--halo.external-url=**********'
      - '--server.port=8090'
    networks:
      - coolify
networks:
  coolify:
    external: true

Exception or Error

No response

Version

v4.0.0-beta.239

[jhenderson]

I've just come across this issue. @andrasbacsai any chance moving this towards the top of the backlog? 😄

[SarkarKurdish]

Same issue here my applications won't join the colify network

[ladderschool]

I am experiencing the same issue, even though I've checked "Connect To Predefined Network" under Advanced for all containers. I have the same yaml file as OP and I've detailed my issue with logs and examples here: https://community.redwoodjs.com/t/deploying-on-coolify/7244/11

This is definitely a production breaking bug. Still happening on v4-300. I just want to say this only occurs during the deployment, once my nodejs app is deployed it's able to see other containers as long as connect to predefined network is on (they're all in the coolify network with the same gateway).

[ladderschool]

During my deployment I ran ifconfig to see what the IP address of the coolify image helper is. This is the output

#9 [api base 4/15] RUN ifconfig
#9 0.297 eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
#9 0.297 inet 172.17.0.2 netmask 255.255.0.0 broadcast 172.17.255.255
#9 0.297 ether [mac] txqueuelen 0 (Ethernet)
#9 0.297 RX packets 4 bytes 370 (370.0 B)
#9 0.297 RX errors 0 dropped 0 overruns 0 frame 0
#9 0.297 TX packets 0 bytes 0 (0.0 B)
#9 0.297 TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

All my other containers that belong to the Coolify network (predefined network) are on 172.18.x.x I think it's clear why this is happening and DNS resolutions doesn't work. The Coolify helper image is in fact on the Coolify network, but the problem occurs right after. During deployment my project is loaded from github and the docker compose is initialized:

[COMMAND] docker exec ak8ksgg bash -c 'SOURCE_COMMIT=hash COOLIFY_BRANCH=main docker compose --env-file /artifacts/ak8ksgg/.env --project-directory /artifacts/ak8ksgg -f /artifacts/ak8ksgg/docker-compose.prod.yml build'
[OUTPUT]
#0 building with "default" instance using docker driver

#1 [api internal] load build definition from Dockerfile
...
#2 [api internal] load metadata for docker.io/library/node:20-bookworm-slim
...
[COMMAND] docker exec ak8ksgg bash -c 'SOURCE_COMMIT=hashCOOLIFY_BRANCH=main docker compose --env-file /artifacts/ak8ksgg/.env --project-directory /artifacts/ak8ksgg -f /artifacts/ak8ksgg/docker-compose.prod.yml build'

This part is right after the helper image finishes and it's where my build process runs inside the Bridge network, NOT the Coolify network even though as OP has described, I've added the Coolify network to my docker compose. This is proven by the ifconfig output I showed above.

docker network ls: coolify:

        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },

bridge:

        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.17.0.0/16",
                    "Gateway": "172.17.0.1"
                }
            ]
        },

I can further confirm that during deployment my settings (same as OP) are running on the bridge network, if you docker inspect the bridge network during deployment you get:

        "Containers": {
            "je6k9y2tia6v4au7fq15a9okp": {
                "Name": "je6k9y2tia6v4au7fq15a9okp",
                "EndpointID": "hash..",
                "MacAddress": "mac..",
                "IPv4Address": "172.17.0.2/16",
                "IPv6Address": ""
            }
        },

This is what my docker-compose looks like for my RedwoodJS app:

version: "3.8"

services:
  api:
    build:
      context: .
      dockerfile: ./Dockerfile
      target: api_serve
    ports:
      - "8911:8911"
    environment:
      - NODE_ENV=
      - DATABASE_URL=
    healthcheck:
      test: curl -f http://api.app.name:8911 || exit 1
      interval: 10s
      start_period: 10s
      timeout: 5s
      retries: 3
    networks:
      coolify: null

  web:
    build:
      context: .
      dockerfile: ./Dockerfile
      target: web_serve
    ports:
      - "8910:8910"
    depends_on:
      api:
        condition: service_healthy
    networks:
      coolify: null

networks:
  coolify:
    name: coolify
    external: true

Notice how I have the same network settings as OP and I'm fairly certain these are not applied at build time based on my testing.

[ladderschool]

Workaround If you have an app that has to insert or do something with your database at build time, it can't reach it as it's currently not joining the coolify network. However, since it's in the bridge network what you can do is add the database container to the bridge network with

docker network connect bridge <db container>

Now during build time you're in the same network and you can execute seeds or anything else.

[atheros]

The problem I see is that coolify include a huge number of predefined services, and many of them include their own postgres DB server. And unless I'm missing something, there is nothing that can be done to share a dedicated postgres instance with them because:

1. you cannot access other networks from docker compose
2. there is no way through the UI to tell: "Hey, I have a postgres instance in this project already, use that instead of spinning up a new one"
3. editing the docker-compose file is kind of buggy (if I get back at trying to use docker compose again, I'll try to narrow down the issues)

[JuanxCursed]

I'm completly lost, or this should be do the work: https://coolify.io/docs/knowledge-base/docker/compose#connect-to-predefined-networks

[Huangwh826]

I'm completly lost, or this should be do the work: https://coolify.io/docs/knowledge-base/docker/compose#connect-to-predefined-networks

Tried it out followed the doc, it works. Not sure if new feature since the issue or always there, might've overlooked it