search

pygmystack / pygmy

the pygmy stack is a container stack for local development
MIT License
25 stars 13 forks source link

[bug] Docker Desktop default Preferences to Use Docker Compose V2 breaks pygmy-go #345

Closed christopher-hopper closed 2 years ago

christopher-hopper commented 2 years ago

Describe the bug Current versions of Docker Desktop ship with Use Docker Compose V2 enabled in Preferences. This setting is not supported by pygmy-go.

Suggest adding it to the Troubleshooting page until Docker Compose V2 support is added in pygmy-go.

To Reproduce Steps to reproduce the behavior:

  1. Go to https://docs.docker.com/desktop/#download-and-install
  2. Follow the instructions to Download and install Docker Desktop
  3. Go to https://pygmy.readthedocs.io/en/master/installation/
  4. Follow the instructions to Download and install pygmy-go
  5. Start pygmy-go with pgymy-go up
  6. Start an Amazee.io Lagoon local Docker Compose environment with docker-compose up -d

Expected behavior Pygmy starts a reverse-proxy, dns forwarder, ssh-agent and mail capture.

Docker Compose then starts, by using the services including the ssh-agent forwarding.

Screenshots

Screen Shot 2021-11-03 pygmy-go amazeeio-ssh-agent error

Output

# pygmy-go up
Successfully started amazeeio-dnsmasq
Successfully started amazeeio-haproxy
Successfully started amazeeio-mailhog
Successfully started amazeeio-ssh-agent
Already connected amazeeio-haproxy to amazeeio-network
Already connected amazeeio-mailhog to amazeeio-network
Already connected amazeeio-ssh-agent to amazeeio-network
VIdentity added: /Users/chopper/.ssh/id_rsa (REDACTED)
 - http://docker.amazee.io/stats (amazeeio-haproxy)
 - http://mailhog.docker.amazee.io (amazeeio-mailhog)

# docker-compose up -d
[+] Running 1/0
 ⠿ Network REDACTED  Created                                                                                                                                                                    0.0s
no such service: amazeeio-ssh-agent

Additional context Add any other context about the problem here.

Exported configuration

Run pygmy export --output output.yml and print the contents from output.yml below:

Defaults: true
Domain: docker.amazee.io
Keys: null
Networks:
  amazeeio-network:
    Attachable: false
    ConfigFrom:
      Network: ""
    ConfigOnly: false
    Containers: null
    Created: "0001-01-01T00:00:00Z"
    Driver: ""
    EnableIPv6: false
    IPAM:
      Config:
      - Gateway: 10.99.99.1
        Subnet: 10.99.99.0/24
      Driver: ""
      Options: null
    Id: ""
    Ingress: false
    Internal: false
    Labels:
      pygmy.name: amazeeio-network
    Name: amazeeio-network
    Options: null
    Scope: ""
Resolvers:
- Data: |
    # Generated by amazeeio pygmy
    nameserver 127.0.0.1
    port 6053
  Enabled: true
  File: docker.amazee.io.conf
  Folder: /etc/resolver
  Name: MacOS Resolver
Services:
  amazeeio-dnsmasq:
    Config:
      AttachStderr: false
      AttachStdin: false
      AttachStdout: false
      Cmd:
      - -A
      - /docker.amazee.io/127.0.0.1
      Domainname: ""
      Entrypoint: null
      Env: null
      Hostname: ""
      Image: andyshinn/dnsmasq:2.83
      Labels:
        pygmy.defaults: "true"
        pygmy.enable: "true"
        pygmy.name: amazeeio-dnsmasq
        pygmy.weight: "13"
      OnBuild: null
      OpenStdin: false
      StdinOnce: false
      Tty: false
      User: ""
      Volumes: null
      WorkingDir: ""
    HostConfig:
      AutoRemove: false
      Binds: null
      BlkioDeviceReadBps: null
      BlkioDeviceReadIOps: null
      BlkioDeviceWriteBps: null
      BlkioDeviceWriteIOps: null
      BlkioWeight: 0
      BlkioWeightDevice: null
      CapAdd:
      - NET_ADMIN
      CapDrop: null
      Cgroup: ""
      CgroupParent: ""
      ConsoleSize:
      - 0
      - 0
      ContainerIDFile: ""
      CpuCount: 0
      CpuPercent: 0
      CpuPeriod: 0
      CpuQuota: 0
      CpuRealtimePeriod: 0
      CpuRealtimeRuntime: 0
      CpuShares: 0
      CpusetCpus: ""
      CpusetMems: ""
      DeviceCgroupRules: null
      Devices: null
      DiskQuota: 0
      Dns: null
      DnsOptions: null
      DnsSearch: null
      ExtraHosts: null
      GroupAdd: null
      IOMaximumBandwidth: 0
      IOMaximumIOps: 0
      IpcMode: private
      Isolation: ""
      KernelMemory: 0
      Links: null
      LogConfig:
        Config: null
        Type: ""
      Memory: 0
      MemoryReservation: 0
      MemorySwap: 0
      MemorySwappiness: null
      NanoCpus: 0
      NetworkMode: ""
      OomKillDisable: null
      OomScoreAdj: 0
      PidMode: ""
      PidsLimit: 0
      PortBindings:
        53/tcp:
        - HostIp: ""
          HostPort: "6053"
        53/udp:
        - HostIp: ""
          HostPort: "6053"
      Privileged: false
      PublishAllPorts: false
      ReadonlyRootfs: false
      RestartPolicy:
        MaximumRetryCount: 0
        Name: unless-stopped
      SecurityOpt: null
      ShmSize: 0
      UTSMode: ""
      Ulimits: null
      UsernsMode: ""
      VolumeDriver: ""
      VolumesFrom: null
    NetworkConfig:
      EndpointsConfig: null
  amazeeio-haproxy:
    Config:
      AttachStderr: false
      AttachStdin: false
      AttachStdout: false
      Cmd: null
      Domainname: ""
      Entrypoint: null
      Env:
      - AMAZEEIO_URL=docker.amazee.io
      Hostname: ""
      Image: amazeeio/haproxy
      Labels:
        pygmy.defaults: "true"
        pygmy.enable: "true"
        pygmy.name: amazeeio-haproxy
        pygmy.network: amazeeio-network
        pygmy.url: http://docker.amazee.io/stats
        pygmy.weight: "14"
      OnBuild: null
      OpenStdin: false
      StdinOnce: false
      Tty: false
      User: ""
      Volumes: null
      WorkingDir: ""
    HostConfig:
      AutoRemove: false
      Binds:
      - /var/run/docker.sock:/tmp/docker.sock
      BlkioDeviceReadBps: null
      BlkioDeviceReadIOps: null
      BlkioDeviceWriteBps: null
      BlkioDeviceWriteIOps: null
      BlkioWeight: 0
      BlkioWeightDevice: null
      CapAdd: null
      CapDrop: null
      Cgroup: ""
      CgroupParent: ""
      ConsoleSize:
      - 0
      - 0
      ContainerIDFile: ""
      CpuCount: 0
      CpuPercent: 0
      CpuPeriod: 0
      CpuQuota: 0
      CpuRealtimePeriod: 0
      CpuRealtimeRuntime: 0
      CpuShares: 0
      CpusetCpus: ""
      CpusetMems: ""
      DeviceCgroupRules: null
      Devices: null
      DiskQuota: 0
      Dns: null
      DnsOptions: null
      DnsSearch: null
      ExtraHosts: null
      GroupAdd: null
      IOMaximumBandwidth: 0
      IOMaximumIOps: 0
      IpcMode: ""
      Isolation: ""
      KernelMemory: 0
      Links: null
      LogConfig:
        Config: null
        Type: ""
      Memory: 0
      MemoryReservation: 0
      MemorySwap: 0
      MemorySwappiness: null
      NanoCpus: 0
      NetworkMode: ""
      OomKillDisable: null
      OomScoreAdj: 0
      PidMode: ""
      PidsLimit: 0
      PortBindings:
        80/tcp:
        - HostIp: ""
          HostPort: "80"
        443/tcp:
        - HostIp: ""
          HostPort: "443"
      Privileged: false
      PublishAllPorts: false
      ReadonlyRootfs: false
      RestartPolicy:
        MaximumRetryCount: 0
        Name: unless-stopped
      SecurityOpt: null
      ShmSize: 0
      UTSMode: ""
      Ulimits: null
      UsernsMode: ""
      VolumeDriver: ""
      VolumesFrom: null
    NetworkConfig:
      EndpointsConfig: null
  amazeeio-mailhog:
    Config:
      AttachStderr: false
      AttachStdin: false
      AttachStdout: false
      Cmd: null
      Domainname: ""
      Entrypoint: null
      Env:
      - MH_UI_BIND_ADDR=0.0.0.0:80
      - MH_API_BIND_ADDR=0.0.0.0:80
      - AMAZEEIO=AMAZEEIO
      - AMAZEEIO_URL=mailhog.docker.amazee.io
      ExposedPorts:
        80/tcp: {}
        1025/tcp: {}
        8025/tcp: {}
      Hostname: ""
      Image: mailhog/mailhog
      Labels:
        pygmy.defaults: "true"
        pygmy.enable: "true"
        pygmy.name: amazeeio-mailhog
        pygmy.network: amazeeio-network
        pygmy.url: http://mailhog.docker.amazee.io
        pygmy.weight: "15"
      OnBuild: null
      OpenStdin: false
      StdinOnce: false
      Tty: false
      User: "0"
      Volumes: null
      WorkingDir: ""
    HostConfig:
      AutoRemove: false
      Binds: null
      BlkioDeviceReadBps: null
      BlkioDeviceReadIOps: null
      BlkioDeviceWriteBps: null
      BlkioDeviceWriteIOps: null
      BlkioWeight: 0
      BlkioWeightDevice: null
      CapAdd: null
      CapDrop: null
      Cgroup: ""
      CgroupParent: ""
      ConsoleSize:
      - 0
      - 0
      ContainerIDFile: ""
      CpuCount: 0
      CpuPercent: 0
      CpuPeriod: 0
      CpuQuota: 0
      CpuRealtimePeriod: 0
      CpuRealtimeRuntime: 0
      CpuShares: 0
      CpusetCpus: ""
      CpusetMems: ""
      DeviceCgroupRules: null
      Devices: null
      DiskQuota: 0
      Dns: null
      DnsOptions: null
      DnsSearch: null
      ExtraHosts: null
      GroupAdd: null
      IOMaximumBandwidth: 0
      IOMaximumIOps: 0
      IpcMode: ""
      Isolation: ""
      KernelMemory: 0
      Links: null
      LogConfig:
        Config: null
        Type: ""
      Memory: 0
      MemoryReservation: 0
      MemorySwap: 0
      MemorySwappiness: null
      NanoCpus: 0
      NetworkMode: ""
      OomKillDisable: null
      OomScoreAdj: 0
      PidMode: ""
      PidsLimit: 0
      PortBindings:
        1025/tcp:
        - HostIp: ""
          HostPort: "1025"
      Privileged: false
      PublishAllPorts: false
      ReadonlyRootfs: false
      RestartPolicy:
        MaximumRetryCount: 0
        Name: unless-stopped
      SecurityOpt: null
      ShmSize: 0
      UTSMode: ""
      Ulimits: null
      UsernsMode: ""
      VolumeDriver: ""
      VolumesFrom: null
    NetworkConfig:
      EndpointsConfig: null
  amazeeio-ssh-agent:
    Config:
      AttachStderr: false
      AttachStdin: false
      AttachStdout: false
      Cmd: null
      Domainname: ""
      Entrypoint: null
      Env: null
      Hostname: ""
      Image: amazeeio/ssh-agent
      Labels:
        pygmy.defaults: "true"
        pygmy.enable: "true"
        pygmy.name: amazeeio-ssh-agent
        pygmy.network: amazeeio-network
        pygmy.output: "false"
        pygmy.purpose: sshagent
        pygmy.weight: "30"
      OnBuild: null
      OpenStdin: false
      StdinOnce: false
      Tty: false
      User: ""
      Volumes: null
      WorkingDir: ""
    HostConfig:
      AutoRemove: false
      Binds: null
      BlkioDeviceReadBps: null
      BlkioDeviceReadIOps: null
      BlkioDeviceWriteBps: null
      BlkioDeviceWriteIOps: null
      BlkioWeight: 0
      BlkioWeightDevice: null
      CapAdd: null
      CapDrop: null
      Cgroup: ""
      CgroupParent: ""
      ConsoleSize:
      - 0
      - 0
      ContainerIDFile: ""
      CpuCount: 0
      CpuPercent: 0
      CpuPeriod: 0
      CpuQuota: 0
      CpuRealtimePeriod: 0
      CpuRealtimeRuntime: 0
      CpuShares: 0
      CpusetCpus: ""
      CpusetMems: ""
      DeviceCgroupRules: null
      Devices: null
      DiskQuota: 0
      Dns: null
      DnsOptions: null
      DnsSearch: null
      ExtraHosts: null
      GroupAdd: null
      IOMaximumBandwidth: 0
      IOMaximumIOps: 0
      IpcMode: private
      Isolation: ""
      KernelMemory: 0
      Links: null
      LogConfig:
        Config: null
        Type: ""
      Memory: 0
      MemoryReservation: 0
      MemorySwap: 0
      MemorySwappiness: null
      NanoCpus: 0
      NetworkMode: ""
      OomKillDisable: null
      OomScoreAdj: 0
      PidMode: ""
      PidsLimit: 0
      PortBindings: null
      Privileged: false
      PublishAllPorts: false
      ReadonlyRootfs: false
      RestartPolicy:
        MaximumRetryCount: 0
        Name: unless-stopped
      SecurityOpt: null
      ShmSize: 0
      UTSMode: ""
      Ulimits: null
      UsernsMode: ""
      VolumeDriver: ""
      VolumesFrom: null
    NetworkConfig:
      EndpointsConfig: null
  amazeeio-ssh-agent-add-key:
    Config:
      AttachStderr: false
      AttachStdin: false
      AttachStdout: false
      Cmd: null
      Domainname: ""
      Entrypoint: null
      Env: null
      Hostname: ""
      Image: amazeeio/ssh-agent
      Labels:
        pygmy.defaults: "true"
        pygmy.discrete: "true"
        pygmy.enable: "true"
        pygmy.name: amazeeio-ssh-agent-add-key
        pygmy.network: amazeeio-network
        pygmy.output: "false"
        pygmy.purpose: addkeys
        pygmy.weight: "31"
      OnBuild: null
      OpenStdin: false
      StdinOnce: false
      Tty: false
      User: ""
      Volumes: null
      WorkingDir: ""
    HostConfig:
      AutoRemove: false
      Binds: null
      BlkioDeviceReadBps: null
      BlkioDeviceReadIOps: null
      BlkioDeviceWriteBps: null
      BlkioDeviceWriteIOps: null
      BlkioWeight: 0
      BlkioWeightDevice: null
      CapAdd: null
      CapDrop: null
      Cgroup: ""
      CgroupParent: ""
      ConsoleSize:
      - 0
      - 0
      ContainerIDFile: ""
      CpuCount: 0
      CpuPercent: 0
      CpuPeriod: 0
      CpuQuota: 0
      CpuRealtimePeriod: 0
      CpuRealtimeRuntime: 0
      CpuShares: 0
      CpusetCpus: ""
      CpusetMems: ""
      DeviceCgroupRules: null
      Devices: null
      DiskQuota: 0
      Dns: null
      DnsOptions: null
      DnsSearch: null
      ExtraHosts: null
      GroupAdd: null
      IOMaximumBandwidth: 0
      IOMaximumIOps: 0
      IpcMode: private
      Isolation: ""
      KernelMemory: 0
      Links: null
      LogConfig:
        Config: null
        Type: ""
      Memory: 0
      MemoryReservation: 0
      MemorySwap: 0
      MemorySwappiness: null
      NanoCpus: 0
      NetworkMode: ""
      OomKillDisable: null
      OomScoreAdj: 0
      PidMode: ""
      PidsLimit: 0
      PortBindings: null
      Privileged: false
      PublishAllPorts: false
      ReadonlyRootfs: false
      RestartPolicy:
        MaximumRetryCount: 0
        Name: ""
      SecurityOpt: null
      ShmSize: 0
      UTSMode: ""
      Ulimits: null
      UsernsMode: ""
      VolumeDriver: ""
      VolumesFrom:
      - amazeeio-ssh-agent
    NetworkConfig:
      EndpointsConfig: null
SortedServices:
- amazeeio-dnsmasq
- amazeeio-haproxy
- amazeeio-mailhog
- amazeeio-ssh-agent
- amazeeio-ssh-agent-add-key
Volumes: {}
fubarhouse commented 2 years ago

I'm happy to review and merge this on the premise of it being a documentation improvement - however it should be noted and clarified that pygmy-go has absolutely no dependence on docker-compose. Docker Compose is a separate application which talks to the Docker API - just like Pygmy.

Individual projects which utilize docker-compose will need to be updated to support docker-compose v2. But, that makes zero difference to pygmy-go and this can be observed in Pygmy's rigorous self-testing - an example of you demonstrating this working from your output:

 - http://docker.amazee.io/stats (amazeeio-haproxy)
 - http://mailhog.docker.amazee.io (amazeeio-mailhog)

I'll start reviewing the PR, but please make sure the issue title reflects the proposed changes that will close this issue as indicated by the reference here: https://github.com/fubarhouse/pygmy-go/issues/345#ref-pullrequest-1043152302.

fubarhouse commented 2 years ago

I've gone and verified the entire process given I've heard a lot of complaints about v2 (not in pygmy, but many other places and people) and the fact that I don't use Docker Desktop on my daily driver. I'll leave this here for referencing later.

Typical resolution I've seen is to update the docker-compose.yml file getting updated for compatability.

 $ docker version
Client:
 Cloud integration: 1.0.17
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:55:20 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.8
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.6
  Git commit:       75249d8
  Built:            Fri Jul 30 19:52:31 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.9
  GitCommit:        e25210fe30a0a703442421b0f60afac609f950a3
 runc:
  Version:          1.0.1
  GitCommit:        v1.0.1-0-g4144b63
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

V2 disabled

image

Enable v2

image

Once applied and restarted, start pygmy

image

Start one or more projects

image

Proof of project 1 resolving correctly

image

Proof of project 2 resolving correctly

image

Retrospectively, proof of V2 being enabled

image

fubarhouse commented 2 years ago

It may be related to how people are using pygmy with the ssh agent. So, I've done some extra testing and opened this one: https://github.com/docker/compose/issues/8874