[BUG] Using multiple compose files with their own includes will only load relatively to the first compose file

MetalArend commented 6 days ago

Description

Running docker compose with two compose files, both including another yaml file, it will only look for the included files relatively to the first loaded compose file.

I would expect both includes to happen based on the path of the compose.yaml file being loaded.

Steps To Reproduce

Create a dir1/compose.yaml file and a dir1/dir1-included.yaml file - add some services to them, and use include in the compose.yaml file to include the dir1-included.yaml file

dir1/compose.yaml

include:
  - ./dir1-included.yaml

services:
  dir1-whoami:
    image: traefik/whoami

dir1/dir1-included.yaml

services:
  dir1-included-whoami:
    image: traefik/whoami

Create a dir2/compose.yaml file and a dir2/dir2-included.yaml file - add some services to them, and use include in the compose.yaml file to include the dir2-included.yaml file

dir2/compose.yaml

include:
  - ./dir2-included.yaml

services:
  dir2-whoami:
    image: traefik/whoami

dir2/dir2-included.yaml

services:
  dir2-included-whoami:
    image: traefik/whoami

Run docker compose -f dir1/compose.yaml -f dir2/compose.yaml up and it will complain about the dir2-included.yamlfile not being found
Run docker compose -f dir2/compose.yaml -f dir1/compose.yaml up and it will complain about the dir1-included.yamlfile not being found
Use a .env file to set the COMPOSE_FILE variable to load dir1/compose.yaml and dir2/compose.yaml to see the same thing happening, depending on the order of files

Compose Version

Docker Compose version v2.27.1

Docker Environment

Client: Docker Engine - Community
 Version:    26.1.4
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 10
  Running: 4
  Paused: 0
  Stopped: 6
 Images: 73
 Server Version: 26.1.4
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d2d58213f83a351ca8f528a95fbd145f5654e957
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-41-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31GiB
 Name: mammi
 ID: ZECH:RGJA:RAZH:K6SZ:L4PM:WQOR:RZQP:F3KE:EMO4:OUFC:RUE6:3CT3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

MetalArend commented 6 days ago

Same issue occurs on Docker Compose version v2.28.1

docker info:

Client: Docker Engine - Community
 Version:    27.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.15.1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.28.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose
  scan: Docker Scan (Docker Inc.)
    Version:  v0.23.0
    Path:     /usr/libexec/docker/cli-plugins/docker-scan

Server:
 Containers: 10
  Running: 0
  Paused: 0
  Stopped: 10
 Images: 73
 Server Version: 27.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-41-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31GiB
 Name: mammi
 ID: ZECH:RGJA:RAZH:K6SZ:L4PM:WQOR:RZQP:F3KE:EMO4:OUFC:RUE6:3CT3
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

ndeloof commented 4 days ago

relative paths always apply based on the "project directory" with is the parent folder for the first compose file. There's unfortunately no syntax to tell compose to consider a path "relative to the current file". Same rule applies to all relative paths, not just include.

docker / compose