search

actions / runner-images

GitHub Actions runner images
MIT License
10.19k stars 3.06k forks source link

Docker compose is incompatible with the new Docker engine #7162

Closed SillyCode closed 1 year ago

SillyCode commented 1 year ago

Description

Following the upgrade to the Docker and Docker Compose and issue #7080.

Currently the latest windows-2019 release is 20230214.1.

Which contains the following Docker related versions: Docker 23.0.1 and Docker Compose V2 2.16.0

Ref: https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Using this Windows-2019 image. We face the issue below: The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.

My docker-compose file

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: my-mssql-image:latest
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"
Operating System
Microsoft Windows Server 2019
10.0.17763
Runner Image
Image: windows-2019
Version: 20230214.1

Looking at the build log. We can see that the issue is reported actually by Choco Docker Compose.

2023-02-19T08:22:09.0368683Z ##[debug]which 'docker-compose'
2023-02-19T08:22:09.0752496Z ##[debug]found: 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:22:10.3052706Z [command]C:\ProgramData\Chocolatey\bin\docker-compose.exe -f D:\a\1\s\Sources\Tests\Docker\docker-compose.yml -f C:\agents\2.217.2\.docker-compose.1676794930282.yml -p SI up -d
2023-02-19T08:22:11.4679494Z Creating network "si_default" with the default driver
2023-02-19T08:22:12.0324536Z Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:22:14.8209567Z I2: Pulling from [i2-si-mssql](my-mssql-image:latest)
2023-02-19T08:32:54.0137344Z The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:54.9175774Z ##[debug]Exit code 1 received from tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:32:54.9177252Z ##[debug]STDIO streams have closed for tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
2023-02-19T08:32:55.0020263Z ##[error]Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:32:55.0021455Z ##[debug]Processed: ##vso[task.issue type=error;]Pulling sql-server-db (***/my-mssql-image:latest)...
2023-02-19T08:32:55.0023243Z ##[error]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0025902Z ##[debug]Processed: ##vso[task.issue type=error;]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0117852Z ##[debug]task result: Failed
2023-02-19T08:32:55.0119880Z ##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Specifically the line ##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Printing the environment variables:

C:\agents\2.217.2\externals\git\cmd;
C:\agents\2.217.2\externals\git\mingw64\bin;
C:\Program Files\MongoDB\Server\5.0\bin;
C:\aliyun-cli;
C:\vcpkg;
C:\cf-cli;
C:\Program Files (x86)\NSIS\;
C:\tools\zstd;
C:\Program Files\Mercurial\;
C:\hostedtoolcache\windows\stack\2.9.3\x64;
C:\cabal\bin;
C:\\ghcup\bin;
C:\Program Files\dotnet;
C:\mysql\bin;
C:\Program Files\R\R-4.2.2\bin\x64;
C:\SeleniumWebDrivers\GeckoDriver;
C:\Program Files (x86)\sbt\bin;
C:\Program Files (x86)\GitHub CLI;
C:\Program Files\Git\bin;
C:\Program Files (x86)\pipx_bin;
C:\npm\prefix;
C:\hostedtoolcache\windows\go\1.17.13\x64\bin;
C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;
C:\hostedtoolcache\windows\Python\3.7.9\x64;
C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;
C:\tools\kotlinc\bin;
C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.362-9\x64\bin;
C:\Program Files\ImageMagick-7.1.0-Q16-HDRI;
C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;
C:\ProgramData\kind;
C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;
C:\Windows\system32;
C:\Windows;
C:\Windows\System32\Wbem;
C:\Windows\System32\WindowsPowerShell\v1.0\;
C:\Windows\System32\OpenSSH\;
C:\ProgramData\Chocolatey\bin;
C:\Program Files\PowerShell\7\;
C:\Program Files\Microsoft\Web Platform Installer\;
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn\;
C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;
C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;
C:\Program Files (x86)\Microsoft SQL Server\160\DTS\Binn\;
C:\Program Files\OpenSSL\bin;
C:\Strawberry\c\bin;
C:\Strawberry\perl\site\bin;
C:\Strawberry\perl\bin;
C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;
C:\Program Files\TortoiseSVN\bin;
C:\Program Files\CMake\bin;
C:\ProgramData\chocolatey\lib\maven\apache-maven-3.9.0\bin;
C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;
C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;
C:\Program Files\nodejs\;
C:\Program Files\Git\cmd;
C:\Program Files\Git\mingw64\bin;
C:\Program Files\Git\usr\bin;
C:\Program Files\GitHub CLI\;
c:\tools\php;
C:\Program Files (x86)\sbt\bin;
C:\SeleniumWebDrivers\ChromeDriver\;
C:\SeleniumWebDrivers\EdgeDriver\;
C:\Program Files\Amazon\AWSCLIV2\;
C:\Program Files\Amazon\SessionManagerPlugin\bin\;
C:\Program Files\Amazon\AWSSAMCLI\bin\;
C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;
C:\Program Files (x86)\Microsoft BizTalk Server\;
C:\Program Files\LLVM\bin;
C:\Users\VssAdministrator\.dotnet\tools;
C:\Users\VssAdministrator\.cargo\bin;
C:\Users\VssAdministrator\AppData\Local\Microsoft\WindowsApps

As can be seen from the environment variable output. There is no mentioning of the Docker path.

Going ahead to get the version of the Choco Docker Compose via:

- task: PowerShell@2
      inputs:
        targetType: 'inline'
        script: |
          C:\ProgramData\Chocolatey\bin\docker-compose.exe version

Outputted the following: docker-compose version 1.29.2, build 5becea4c

Choco Docker Compose for reference: https://community.chocolatey.org/packages/docker-compose

To summarize. It seems that the Docker Compose is installed via Choco package. And it is currently supports old version. Hence it seems to me, why we have incompatibility issue. In spite of having latest Docker Compose version listed on the image.

Platforms affected

Runner images affected

Image version and build link

Operating System Microsoft Windows Server 2019 10.0.17763 Runner Image Image: windows-2019 Version: 20230214.1

Image release: https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Is it regression?

yes. Version 20230129.2

Expected behavior

The Docker container with Docker Compose should run without having incompatibility issue between the Docker compose and Docker engine

Actual behavior

Currently, we have the following error generated when trying to run Docker compose

2023-02-19T08:32:55.0023243Z ##[error]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0025902Z ##[debug]Processed: ##vso[task.issue type=error;]The current Compose file version is not compatible with your engine version. Please upgrade your Compose file to a more recent version, or set a COMPOSE_API_VERSION in your environment.
2023-02-19T08:32:55.0117852Z ##[debug]task result: Failed
2023-02-19T08:32:55.0119880Z ##[error]The process 'C:\ProgramData\Chocolatey\bin\docker-compose.exe' failed with exit code 1

Repro steps

Below are the steps that reproduce the issue:

- task: DockerCompose@0
      displayName: Spin up container
      inputs:
        containerregistrytype: 'Container Registry'
        dockerRegistryEndpoint: 'Schema-BuildReg'
        dockerComposeFile: 'Sources/Tests/Docker/docker-compose.yml'
        action: 'Run a Docker Compose command'
        dockerComposeCommand: 'up'
        arguments: '-d'

My docker-compose file

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: my-mssql-image:latest
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"
mikhailkoliada commented 1 year ago

Hello @SillyCode, we will take a look

sergei-pyshnoi commented 1 year ago

Hello @SillyCode, we wasn`t able to reproduce this error using docker-compose file with version 3.9. For testing we used more simplified docker compose configuration with windows/nanoserver image (unfortunately I did not find information about image my-mssql-image:latest that you use in your example), and build was successful. I suggest the problem can be with specified container image that you use. If problem still exist, please provide more detailed repro steps (for example minimum repo with ready pipeline and docker-compose file).

docker-compose.yml

version: "3.9"
services:
  sql-server-db:
    container_name: my-mssql
    image: mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
    ports:
      - "1433:1433"
    environment:
      SA_PASSWORD: "***************"
      ACCEPT_EULA: "Y"

Starting: Initialize job logs

Starting: Initialize job
Agent name: 'Hosted Agent'
Agent machine name: 'WIN-D0QP9LQROBF'
Current agent version: '2.217.2'
Operating System
Microsoft Windows Server 2019
10.0.17763
Datacenter
Runner Image
Image: windows-2019
Version: 20230214.1
Included Software: https://github.com/actions/runner-images/blob/win19/20230214.1/images/win/Windows2019-Readme.md
Image Release: https://github.com/actions/runner-images/releases/tag/win19%2F20230214.1

Spin up container job log

Creating network "azuritetest_default" with the default driver
Pulling sql-server-db (mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64)...
1809-KB5022840-amd64: Pulling from windows/nanoserver
Digest: sha256:4fa54f9de5b106a5695b3227a1cceb2a35fee92b0e922b4010691a67d600cbd2
Status: Downloaded newer image for mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
Creating my-mssql ... 

Creating my-mssql ... done

Finishing: Spin up container
SillyCode commented 1 year ago

Hi sergei-pyshnoi, Thank you for having some time for this.

I use a private private registry image that is based on mssql 2019. It was working fine until the Docker upgrade. Have to mention that no changes to pipeline were made since.

Can you tell which service runs the Docker-compose in the log? In my case it is located in Chocolatey\bin

##[debug]which 'docker-compose'
##[debug]found: 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
sergei-pyshnoi commented 1 year ago

@SillyCode 

C:\ProgramData\Chocolatey\bin\docker-compose.exe -f D:\a\1\s\docker-compose.yaml -f C:\agents\2.217.2\.docker-compose.1676992147189.yml -p AzuriteTest up -d
Creating network "azuritetest_default" with the default driver
Pulling sql-server-db (mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64)...
1809-KB5022840-amd64: Pulling from windows/nanoserver
Digest: sha256:4fa54f9de5b106a5695b3227a1cceb2a35fee92b0e922b4010691a67d600cbd2
Status: Downloaded newer image for mcr.microsoft.com/windows/nanoserver:1809-KB5022840-amd64
Creating my-mssql ... 

Creating my-mssql ... done

##[debug]Exit code 0 received from tool 'C:\ProgramData\Chocolatey\bin\docker-compose.exe'
SillyCode commented 1 year ago

I am at lost to why does it work. You can also see mathias-kogler reply

And I can confirm that his workaround works. I believe there is something wrong with the DockerCompose@0 task. It should take the latest Docker-Compose utility instead of the old one from Choco, that is incompatible

sergei-pyshnoi commented 1 year ago

Hello @SillyCode . You are right, the DockerCompose@0 use version from C:\ProgramData\Chocolatey\bin\ . You can create issue in tasks repo about this, because its out of scope of runner-images repo.

SillyCode commented 1 year ago

Thank you @sergei-pyshnoi . I will.