Docker Compose V2 does not seem to support "context"

[nsteinmetz]

Description

context seems not supported with docker compose v2

Previously, I could run:

docker context use <remote context>
docker-compose <command>

or:

docker-compose --context <remote context> <command>

With 2 servers having:

• docker-ce 20.10.14
• docker-compose-plugin 2.3.3

From Server A with context foo being set to connect to server B:

docker context create foo --description "Server B" --docker "host=ssh://user@host"

• docker --context foo ps => ✅
• docker --context foo compose ps => ❌

Describe the results you received:

No output as if no containers were running:

$ docker --context foo compose -f docker-compose.staging.yml ps -a
NAME                COMMAND             SERVICE             STATUS              PORTS
$

Describe the results you expected:

docker compose should be context aware and allow to run remote commands.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.3.3

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.1-docker)
  compose: Docker Compose (Docker Inc., v2.3.3)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 4
 Server Version: 20.10.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3df54a852345ae127d1fa3092b95168e4a88e2f8
 runc version: v1.0.3-0-gf46b6ba
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.10.0-13-cloud-amd64
 Operating System: Debian GNU/Linux 11 (bullseye)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 6.627GiB
 Name: flovea-gitlab
 ID: VFCE:23YZ:GRZT:MSCZ:GNQG:FU7T:DCXJ:PS5T:VVRZ:325Y:KOYO:SMK2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

Debian 11.x Servers - up to date.

[glours]

Can you upgrade to the latest version of Compose v2.4.1 and check if the issue isn't already solved? As mentioned on twitter with @ndeloof we can't reproduce with the latest version

And can you add the exact error message you get when you try to use the --context flag with Compose, please?

[nsteinmetz]

Just upgraded to 2.4.1 on server A (gitlab) to connect to server B (tooling ; compose version unchanged on this host):

gitlab-runner@gitlab:~/builds/1WShLTsf/0/projectA/core$ docker compose version
Docker Compose version v2.4.1
gitlab-runner@gitlab:~/builds/1WShLTsf/0/projectA/core$ docker --context tooling compose  ps -a
NAME                COMMAND             SERVICE             STATUS              PORTS
gitlab-runner@gitlab:~/builds/1WShLTsf/0/projectA/core$ docker --context tooling compose ps
NAME                COMMAND             SERVICE             STATUS              PORTS
gitlab-runner@gitlab:~/builds/1WShLTsf/0/projectA/core$ docker --context tooling compose -f docker-compose.yml ps
NAME                COMMAND             SERVICE             STATUS              PORTS
gitlab-runner@gitlab:~/builds/1WShLTsf/0/projectA/core$

[ndeloof]

can you please check:

docker --context tooling ps
docker --context tooling inspect <some compose container from previous command> | grep com.docker.compose.project
docker --context tooling compose ls

[nsteinmetz]

Hi,

Below the results:

 docker --context tooling ps
CONTAINER ID   IMAGE                                                           COMMAND                  CREATED       STATUS                 PORTS                NAMES
eaa9cce59f14   warp10io/warp10:2.10.1                                          "/bin/sh -c ${WARP10…"   3 hours ago   Up 3 hours             8080-8081/tcp        staging-warp10-1
7ccea54ee733   code.boite.fr:5050/project/core/project-core:0.4.3.c4ed1381    "uvicorn app.main:ap…"   4 hours ago   Up 4 hours             8000/tcp             staging-api-1
a21896bfd99a   postgres:14.2                                                   "docker-entrypoint.s…"   4 hours ago   Up 4 hours (healthy)   5432/tcp             staging-postgres-1
058be71373d2   passbolt/passbolt:3.5.0-ce-non-root                             "/usr/bin/wait-for.s…"   2 weeks ago   Up 2 weeks             4433/tcp, 8080/tcp   passbolt-passbolt-1
2217329af435   code.boite.fr:5050/project/docker-images/passbolt-backup:0.1   "crond -f"               2 weeks ago   Up 2 weeks                                  passbolt-backup-1
820b1147dc85   mariadb:10.7                                                    "docker-entrypoint.s…"   2 weeks ago   Up 2 weeks             3306/tcp             passbolt-db-1
78662a53b56b   code.boite.fr:5050/project/docker-images/mlflow:1.24.0         "/bin/sh -c 'exec ml…"   3 weeks ago   Up 3 weeks             5000/tcp             mlflow-mlflow-1

Only the staging-* containers are part of my compose file and there could be also the mlflow-* and the passbolt-* ones.

docker --context tooling inspect eaa9cce59f14  | grep com.docker.compose.project
                "com.docker.compose.project": "staging",
                "com.docker.compose.project.config_files": "/home/debian/project/staging/docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/debian/project/staging",

and the last one:

docker --context tooling compose ls
NAME                STATUS              CONFIG FILES
mlflow              running(1)          /home/debian/datascience/mlflow/docker-compose.yml
passbolt            running(3)          /home/debian/team/passbolt/docker-compose.yml
staging             running(3)          /home/debian/project/staging/docker-compose.yml

compose projects were deployed so far locally on target server, like:

From gitlab server, in a gitlab-ci task:

ssh debian@tooling "docker compose -f  /home/debian/project/staging/docker-compose.yml up -d"

[ndeloof]

ok, so docker --context tooling compose ls was able to detect those (remote) container being part of staging project.

Can you please check docker --context tooling compose -p staging ps?

[nsteinmetz]

Seems that the issue is about project name indeed (parent dir on gitlab side is core) and not staging as on the target server :

docker --context tooling compose -p staging ps
NAME                 COMMAND                  SERVICE             STATUS              PORTS
staging-api-1        "uvicorn app.main:ap…"   api                 running             8000/tcp
staging-postgres-1   "docker-entrypoint.s…"   postgres            running (healthy)   5432/tcp
staging-warp10-1     "/bin/sh -c ${WARP10…"   warp10              running             8080-8081/tcp

Didn't think about it - even if I hit this issue a long time ago but did not remember it when re-using compose for this project...

Unless you see a remaining bug, I think we're done and we can close the issue.

Thanks for your help :)