search

docker / compose

Define and run multi-container applications with Docker
https://docs.docker.com/compose/
Apache License 2.0
33.84k stars 5.21k forks source link

docker-compose '-H'/'--host' missing in docker compose #8801

Closed Tea-Ice closed 1 year ago

Tea-Ice commented 3 years ago

Description

We use docker-compose with the -H/--host as part of our standard build pipeline, with the latest update of docker on Mac, it forced the use of the new CLI tool and broke all of our deploys.

Steps to reproduce the issue:

  1. Run any command with -H/--host to use a remote socket
  2. Watch it break

Describe the results you received: Gives an unknown flag error

Describe the results you expected: It to use the remote socket

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.0.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Build with BuildKit (Docker Inc., v0.6.3)
  compose: Docker Compose (Docker Inc., v2.0.0)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 44
 Server Version: 20.10.8
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e25210fe30a0a703442421b0f60afac609f950a3
 runc version: v1.0.1-0-g4144b63
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.47-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 3.842GiB
 Name: docker-desktop
 ID: AWDL:LZ3J:PVVP:L26Q:I7PK:LLBM:BOSL:W6HC:DGAG:YPN5:MYJD:KJBH
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 42
  Goroutines: 43
  System Time: 2021-10-15T08:36:20.537155322Z
  EventsListeners: 3
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

bendhayer commented 3 years ago

We have the same issue on Linux. docker -H "ssh://user@address" compose fails with the message unknown docker command: "compose compose" after displaying the command usage documentation.

Output of docker compose version:

Docker Compose version v2.0.1

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  compose: Docker Compose (Docker Inc., v2.0.1)
  scan: Docker Scan (Docker Inc., v0.8.0)

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 23
 Server Version: 20.10.9
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 5b46e404f6b9f661a205e28d59c982d3634148f8
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
 Kernel Version: 5.4.0-81-generic
 Operating System: Ubuntu 20.04.3 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 48
 Total Memory: 62.69GiB
 Name: cdsuite-par-ht
 ID: 6DRJ:QFJZ:RK3R:GQCZ:FIK7:5ZNT:JKDQ:M4PN:LA7V:IKSH:7ALC:P4UB
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
ssipos90 commented 2 years ago

does anyone know a workaround for the moment?

ndeloof commented 2 years ago

Can you please try running /Applications/Docker.app/Contents/Resources/cli-plugins/docker-compose -H .... command?

ssipos90 commented 2 years ago

@ndeloof seems to be working running the compose binary.

ssipos90 commented 2 years ago 
$ /usr/lib/docker/cli-plugins/docker-compose -H 'ssh://....' -p services ps
NAME                   COMMAND                  SERVICE             STATUS              PORTS
services_mongo42_1     "docker-entrypoint.s…"   mongo42             running             27017/tcp
...
darklow commented 2 years ago

@ssipos90 Only workaround I found (not to change command line) is downgraded back to Docker Desktop 4.1.1. Starting from 4.2.0 already broken and seems on 4.2.0 compose v2.1.1 was added where something obviously changed.

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

ssipos90 commented 2 years ago

@ssipos90 Only workaround I found (not to change command line) is downgraded back to Docker Desktop 4.1.1. Starting from 4.2.0 already broken and seems on 4.2.0 compose v2.1.1 was added where something obviously changed.

we aren't using docker machine.

stale[bot] commented 2 years ago

This issue has been automatically marked as not stale anymore due to the recent activity.

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Tea-Ice commented 1 year ago

Not stale, still very much an issue.

stale[bot] commented 1 year ago

This issue has been automatically marked as not stale anymore due to the recent activity.

ndeloof commented 1 year ago

@Tea-Ice could you please provide more details on your installation? I can't reproduce this issue on OSX

~ docker-compose -H /var/run/docker.sock version
Docker Compose version v2.15.0
Tea-Ice commented 1 year ago

@ndeloof Also on OSX, think I might have been unclear in my original post. Same as @bendhayer, we use the HOST variable to run commands remotely:

$ docker compose -f compose/alpha.yml -H ssh://$user@$host pull
unknown shorthand flag: 'H' in -Ha.yml -H ssh://$user@$host pull

$ docker-compose -f compose/alpha.yml -H ssh://$user@$host pull
unknown shorthand flag: 'H' in -Ha.yml -H ssh://$user@$host pull

If I switch back to compose v1, it works perfectly.

My environment:

$ docker info
Client:er info                                                                                                                                                                                                                                                                                                                                                                                       ─╯
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.9.1)
  compose: Docker Compose (Docker Inc., v2.13.0)
  dev: Docker Dev Environments (Docker Inc., v0.0.5)
  extension: Manages Docker extensions (Docker Inc., v0.2.16)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.22.0)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 7
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.49-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 6
 Total Memory: 3.841GiB
 Name: docker-desktop
 ID: 4PVS:WGOU:7NAE:KC7R:ZSLL:XQ7E:CUV7:IT54:PW2K:3E7R:YJPC:ZF2T
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 46
  Goroutines: 50
  System Time: 2023-01-09T09:45:12.329526822Z
  EventsListeners: 5
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5000
  127.0.0.0/8
 Live Restore Enabled: false
ndeloof commented 1 year ago

-H option has to be set on the top-level docker command, not after compose subcommand (same as any other docker command by the way): docker -H ssh://$user@$host pull compose -f compose/alpha.yml

Tea-Ice commented 1 year ago

Why the difference in behaviour across v1 and v2?

BTW, the command wasn't completely correct. Had to modify it to work:

$ docker -H ssh://$user@$host compose -f compose/$env.yml pull

Thanks. Can finally switch to compose v2.

ndeloof commented 1 year ago

While we could support -H passed after the command, as demonstrated in my PR^, this comes with side effects (see https://github.com/docker/compose/issues/8648) so better not try to be too clever.