search

actions / runner-images

GitHub Actions runner images
MIT License
10.13k stars 3.05k forks source link

[Ubuntu, Windows] Docker Compose v1 will be removed from images on April, 1 #9557

Closed mikhailkoliada closed 7 months ago

mikhailkoliada commented 7 months ago

Breaking changes

Docker compose v1 will be removed from Ubuntu & Windows images.

Target date

The images rollout process will start on April, 1 and take 3-4 days.

The motivation for the changes

Docker compose v1 has been deprecated since July 2023 and customers are encouraged to migrate to v2, apart from that, as per our support policy it has been more that 6 months since the deprecation announcement so we would like to finally remove it.

Possible impact

If your pipelines or workflows still depend on docker compose v1 they might be broken.

Platforms affected

Runner images affected

Mitigation ways

The last release of the v1 branch is still available on GitHub and can be downloaded and installed manually in runtime for both Ubuntu and Windows.

Loki-Afro commented 7 months ago

@mikhailkoliada

from https://docs.docker.com/compose/migrate/

With Docker Desktop, Compose V2 is always accessible as docker compose. Additionally, the Use Compose V2 setting is turned on by default, which provides an alias from docker-compose.

will you provide such an alias as well? as this is basically the breaking point for a lot of projects :)

abonander commented 7 months ago

A more visible notice would have been nice. I didn't find out about this until CI for my repo broke, literally in the middle of a workflow. I was absolutely baffled to see the command working on some runs and failing on others.

abonander commented 7 months ago

This wasn't even announced on the blog: https://github.blog/changelog/

sayhiben commented 7 months ago

Hi there - we are similarly scrambling to fix old workflows. @mikhailkoliada will you please provide a link to where the deprecation notice was posted so that we can avoid issues in the future?

jdkent commented 7 months ago

this isn't a perfect search, but it looks like this change may impact a few repos yet:

https://github.com/search?q=path%3A.github%2Fworkflows+AND+%28%22docker-compose+build%22+OR+%22docker-compose+pull%22%29&type=code

sayhiben commented 7 months ago

this isn't a perfect search, but it looks like this change may impact a few repos yet:

IMG_8294

sayhiben commented 7 months ago

Riffing on @abonander's search - seems like we could capture more with this regex content search:

path:.github/workflows AND (content:/.*docker-compose\s+\b.*/)

image

That regex is:

Example results:

It's going to over-index on stuff like comments and docs that use the term. It's also going to miss any docker compose aliases and probably some other cases. Regardless, I'd guess it's a good proxy of actual number of lines/files/whatever the search count represents

We could improve that estimate by enumerating all v1 commands in the regex, but we still won't know how many of the workflows are active or any idea of how many private repositories (I.e. businesses) are impacted

On a personal level, I totally agree it's time to get off of Docker Compose v1. At the same time, my team was completely unaware of this change, and it impacted our ability to deploy.

My ask is that deprecations are announced more than once and in a place that faces the average GitHub customer, such as the changelog or on a calendar such as the roadmap

Six months is short for enterprises, but workable. Shouting a deprecation notice into the void (I.e., this Issue/thread/repo) ensures it's going unnoticed by us until it impacts my team and our customers. Please at least announce deprecations in the same manner as actions/checkout's Node upgrade spam

cleor41 commented 7 months ago

@mikhailkoliada

from https://docs.docker.com/compose/migrate/

With Docker Desktop, Compose V2 is always accessible as docker compose. Additionally, the Use Compose V2 setting is turned on by default, which provides an alias from docker-compose.

will you provide such an alias as well? as this is basically the breaking point for a lot of projects :)

This seems like the solution - unless there's something preventing it?

sayhiben commented 7 months ago

@mikhailkoliada from https://docs.docker.com/compose/migrate/

With Docker Desktop, Compose V2 is always accessible as docker compose. Additionally, the Use Compose V2 setting is turned on by default, which provides an alias from docker-compose.

will you provide such an alias as well? as this is basically the breaking point for a lot of projects :)

This seems like the solution - unless there's something preventing it?

I agree and would add that it should also post a deprecation notice to the Workflow Run page whenever this alias is accessed

For most users, docker-compose run and docker compose run will do the same thing, but for some, there are differences that matter

bbaassssiiee commented 7 months ago

April Fools?

fbuser1 commented 7 months ago

Wow, my entire company is blown away after this change 😡

Why didn't Microsoft issue any warnings months before the change? For example: 'This task will be deprecated, please migrate to docker-compose v2 before the 1st of April.'

Currently, I have to update about 500 microservices names from 'Company.Project.ServiceName' to 'company-project-servicename', which affects a lot of components/configs/DockerFiles entrypoints

mikhailkoliada commented 7 months ago

Hi all, looks like Azure DevOps customers were not prepared for this change and while Actions customers need to only modify their pipelines to mostly switch docker compose from docker-compose in ADO it is much harder as the actual compose path is being managed by the task which is hard to modify. We are looking into the revert now, expected ETA is ~6-8h

derberg commented 7 months ago

sorry, what revert means? docker compose will now not work and we need to switch back to docker-compose? please don't 🙏🏼

sayhiben commented 7 months ago

Hi all, looks like Azure DevOps customers were not prepared for this change and while Actions customers need to only modify their pipelines to mostly switch docker compose from docker-compose in ADO it is much harder as the actual compose path is being managed by the task which is hard to modify. We are looking into the revert now, expected ETA is ~6-8h

@mikhailkoliada, it's disappointing to hear your only response is to manage Azure DevOps customers rather than consider the huge number of impacted Actions customers, ignoring our concerns on the basis of "need only to switch to docker compose", which is patently untrue

Allow me to ask more directly:

How does GitHub plan to make deprecation notices visible to its customers moving forward?

sayhiben commented 7 months ago

sorry, what revert means? docker compose will now not work and we need to switch back to docker-compose? please don't 🙏🏼

I'd expect both to work after the revert; v2 has been available for a while now

AnthonyDewhirst commented 7 months ago

What was the expected migration path for those using DockerCompose@0? will there be a @1 or does Docker@2 already handle "compose" as a command (the docs don't state that it does: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/docker-v2?view=azure-pipelines&tabs=yaml#inputs) or will there then need to be a Docker@3 that does have this? Where have you outlined the migration path for the people using these tasks?

fionnachan commented 7 months ago

Hi @mikhailkoliada, can you alias docker-compose to use v2 in the runner images and add a log to say it's deprecated and would be removed by this date?

This GH issue doesn't seem to have reached all of its target audience, and as you can see, people are quite disappointed to see their ci started breaking all of a sudden.

It is particularly difficult for people with github actions dependencies that use docker-compose, and can't immediately change the usage in the dependencies.

SwapnaVersion1 commented 7 months ago

pos

Hi, we are also facing issues using ADO task. Could you please let us know the workaround time? We have priority testing blocked due to this.

laurencenairne-work commented 7 months ago

pos

Hi, we are also facing issues using ADO task. Could you please let us know the workaround time? We have priority testing blocked due to this.

Hey, I have it working in our pipelines by setting two input properties in the DockerCompose@0 task: projectName and dockerComposePath. This (I believe) makes it use docker compose v2. The projectName is required to make our repository name lower case, but your mileage may vary depending on how you name your repos.

- task: DockerCompose@0
          displayName: Build Docker Images
          inputs:
            action: Build services
            azureSubscriptionEndpoint: $(SERVICE_CONNECTION_UAT)
            azureContainerRegistry: $(CONTAINER_REGISTRY)
            projectName: ${{ lower(variables['Build.Repository.Name']) }}
            dockerComposePath: '/usr/libexec/docker/cli-plugins/docker-compose'
            dockerComposeFile: build-docker-compose.yml
SwapnaVersion1 commented 7 months ago

@ laurencenairne-work ##[error]invalid project name "test/test-testrepo-repo": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number

I am seeing this error.

SwapnaVersion1 commented 7 months ago

pos

Hi, we are also facing issues using ADO task. Could you please let us know the workaround time? We have priority testing blocked due to this.

Hey, I have it working in our pipelines by setting two input properties in the DockerCompose@0 task: projectName and dockerComposePath. This (I believe) makes it use docker compose v2. The projectName is required to make our repository name lower case, but your mileage may vary depending on how you name your repos.

- task: DockerCompose@0
          displayName: Build Docker Images
          inputs:
            action: Build services
            azureSubscriptionEndpoint: $(SERVICE_CONNECTION_UAT)
            azureContainerRegistry: $(CONTAINER_REGISTRY)
            projectName: ${{ lower(variables['Build.Repository.Name']) }}
            dockerComposePath: '/usr/libexec/docker/cli-plugins/docker-compose'
            dockerComposeFile: build-docker-compose.yml

[error]invalid project name "hmcts/vh-video-api": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number

laurencenairne-work commented 7 months ago

Yep, that was the "your mileage may vary" bit. compose v2 doesn't like project names that contain special characters and uppercase letters.

So you will need to transform your project name into something without those. In my example above, our repository name doesn't contain special characters, only uppercase letters, so this works for us.

To be specific, you have a / in your repo name, so you'll have to transform that name to remove the slash when running that task (in the same way you've used lower to remove uppercase.

mikhailkoliada commented 7 months ago

Amended now, I will contact ado later to figure out if they are going to update the task in first place before we proceed with the actual retirement of v1

laurencenairne-work commented 7 months ago

@mikhailkoliada I'm glad that you've amended something, but it's still not clear to everyone here what exactly was amended. There are a few comments above making suggestions and one question about what "revert" means.

Any chance of you (or someone else) answering those before this ticket is turned to dust in the back of your mind?

SwapnaVersion1 commented 6 months ago

Hi @mikhailkoliada , Do we have solution for ADO tasks DockerCompose@0? Can we use upgraded task rather than using the work around solution?

fabiante commented 6 months ago

We have rewritten all of our DockerCompose@ commands to simply use Bash.

-    - task: DockerCompose@0
+    - task: Bash@3
       displayName: 'OpenAPI Linter: Run Redocly CLI Linter'
       inputs:
-        action: 'Run a Docker Compose command'
-        dockerComposeFile: devops/azure/docker-compose.yml
-        dockerComposeCommand: >
+        targetType: inline
+        script: >
+          docker compose -f devops/azure/docker-compose.yml
           exec -T
           --workdir /var/app
           application
           ...

You may be lucky enough that Copilot / similar tools will quickly rewrite those tasks

SwapnaVersion1 commented 6 months ago

We have rewritten all of our DockerCompose@ commands to simply use Bash.

-    - task: DockerCompose@0
+    - task: Bash@3
       displayName: 'OpenAPI Linter: Run Redocly CLI Linter'
       inputs:
-        action: 'Run a Docker Compose command'
-        dockerComposeFile: devops/azure/docker-compose.yml
-        dockerComposeCommand: >
+        targetType: inline
+        script: >
+          docker compose -f devops/azure/docker-compose.yml
           exec -T
           --workdir /var/app
           application
           ...

You may be lucky enough that Copilot / similar tools will quickly rewrite those tasks

Thank you. I am looking for ADO task as we have used it across and should be able to use next version. Hopefully we have a fix. otherwise I will have to re-write to bash

lkfortuna commented 6 months ago

👋 Hey all, I am a PM on the Actions team here at GitHub. For those who didn't see the changelog around this, I am reposting the information here as well.

tl;dr we added Docker compose v1 back into the Ubuntu images for now. We will officially deprecate Docker Compose v1 on July 9.

Thank you for your patience and understanding as we worked to address this issue. I appreciate the difficulty this has caused many of you in the community. We do strive to announce major breaking changes via the changelog, and we did not do that here initially, so that was a miss on our end.

Text from the changelog below for reference:

Docker Compose v1 has been deprecated as of July 2023. All customers utilizing Compose v1 on GitHub-hosted runners are encouraged to migrate to Compose v2. Per GitHub’s support policy we will remove this tool from our GitHub managed runner images effective July 9, 2024.

To avoid breaking changes, customers will need to update their Actions workflow files from using docker-compose to docker compose. After July 9, workflows will begin to fail that are using the previous syntax. Customers are advised to review the migration instructions to ensure they are making all the changes required.

For more information on GitHub managed images, see the runner-images repository.

fabiante commented 6 months ago

Thank you for adding more context.

Maybe this I am totally of with this: We have noticed the breaking change in our Azure DevOps runners (hosted by Mircosoft). That is where our pipelines broke.

Is my understanding correct that the Azure DevOps Runners processing our CI jobs are related to the runners behind GitHub Actions? Excuse me if this is an unrelated topic.

SwapnaVersion1 commented 5 months ago

👋 Hey all, I am a PM on the Actions team here at GitHub. For those who didn't see the changelog around this, I am reposting the information here as well.

tl;dr we added Docker compose v1 back into the Ubuntu images for now. We will officially deprecate Docker Compose v1 on July 9.

Thank you for your patience and understanding as we worked to address this issue. I appreciate the difficulty this has caused many of you in the community. We do strive to announce major breaking changes via the changelog, and we did not do that here initially, so that was a miss on our end.

Text from the changelog below for reference:

Docker Compose v1 has been deprecated as of July 2023. All customers utilizing Compose v1 on GitHub-hosted runners are encouraged to migrate to Compose v2. Per GitHub’s support policy we will remove this tool from our GitHub managed runner images effective July 9, 2024. To avoid breaking changes, customers will need to update their Actions workflow files from using docker-compose to docker compose. After July 9, workflows will begin to fail that are using the previous syntax. Customers are advised to review the migration instructions to ensure they are making all the changes required. For more information on GitHub managed images, see the runner-images repository.

Does this mean we can amend our ADO CI pipeline to use Docker @ v2? Could you please confirm?