Open dhavemeyer-symplr opened 2 months ago
Hi @dhavemeyer-symplr Could you please share the complete debug logs of the pipeline by adding variable system.debug to "true"?
The DockerCompose@0 task defaults projectName to '$(Build.Repository.Name)'. However, GitHub repositories have a forward slash (/) in their name, and the forward slash is an invalid property for projectName when calling docker compose. I'm not sure why this only just broke, but manually specifying a project name that does not include the slash works.
Could you please elaborate your query?
Hi,
All our azure devops pipelines have started to fail with the following message:
Starting: Build services
Task : Docker Compose Description : Build, push or run multi-container Docker applications. Task can be used with Docker or Azure Container registry. Version : 0.240.2 Author : Microsoft Corporation Help : https://aka.ms/azpipes-docker-compose-tsg
Finishing: Build services
This has been working until now, all our different repositories are named like this
This is the task we're using:
- task: DockerCompose@0
displayName: 'Build services'
inputs:
containerregistrytype: 'Azure Container Registry'
dockerComposeFile: 'docker-compose.yml'
dockerComposeFileArgs: |
BuildConfiguration=$(BuildConfiguration)
ASPNETCORE_ENVIRONMENT='Production'
ImageName=$(imageName)
TOKEN_NPM=$(TOKEN_NPM)
action: 'Build services'
additionalImageTags: '$(tag)'`
Hi,
All our azure devops pipelines have started to fail with the following message:
Starting: Build services
Task : Docker Compose Description : Build, push or run multi-container Docker applications. Task can be used with Docker or Azure Container registry. Version : 0.240.2 Author : Microsoft Corporation Help : https://aka.ms/azpipes-docker-compose-tsg
[error]invalid project name "Company.Project.Component": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number ##[error]The process '/usr/bin/docker' failed with exit code 15 Finishing: Build services
This has been working until now, all our different repositories are named like this .. with PascalCasing and it was never a problem.
This is the task we're using:
- task: DockerCompose@0 displayName: 'Build services' inputs: containerregistrytype: 'Azure Container Registry' dockerComposeFile: 'docker-compose.yml' dockerComposeFileArgs: | BuildConfiguration=$(BuildConfiguration) ASPNETCORE_ENVIRONMENT='Production' ImageName=$(imageName) TOKEN_NPM=$(TOKEN_NPM) action: 'Build services' additionalImageTags: '$(tag)'`
We are already working on this issue. Will keep here posted the latest updates.
I'm also experiencing this issue.
Failing on a project name of e.g. ProjectName_1.2.3
As a workaround I've set the "projectName" property of the DockerCompose Task to use lowercase and underscores, e.g. projectname_1_2_3
The documentation implies that upper-case and periods can be used as per:
-------Docker Compose v0 task---------------- projectName - Project Name string. Default value: $(Build.Repository.Name).
Specifies the project name to use by default to name images and containers.
-------DevOps: Create a repository---------------- TF401025: 'repoName' is not a valid name for a Git repository. A repository name cannot: contain more than 64 characters, contain Unicode control characters or surrogate characters, contain any of the following characters: / : \ ~ & % ; @ ' " ? < > | # $ * } { , + = [ ], start with an underscore ( _ ), start or end with a period ( . ), or be a system reserved name
Hi @dhavemeyer-symplr Could you please share the complete debug logs of the pipeline by adding variable system.debug to "true"?
For the time being I cannot. It looks like the feature flag that caused this has since been disabled in my ADO organization.
##[debug]No pattern found in docker compose filepath parameter
##[debug]which 'docker'
##[debug]found: 'C:\Windows\system32\docker.exe'
**##[debug]Feature flag USE_DOCKER_COMPOSE_V2_COMPATIBLE_MODE not found. Returning false as default.**
##[debug]dockerComposePath=undefined
##[debug]which 'docker-compose'
##[debug]found: 'C:\ProgramData\docker-compose\docker-compose.exe'
##[debug]dockerComposeFile=D:\a\1\s\docker-compose.yml
##[debug]cwd=D:\a\1\s
To be clear though, this will still be an issue in any environment where that feature flag is enabled and it sounds like this was rolled out to many other organizations as well.
The DockerCompose@0 task defaults projectName to '$(Build.Repository.Name)'. However, GitHub repositories have a forward slash (/) in their name, and the forward slash is an invalid property for projectName when calling docker compose. I'm not sure why this only just broke, but manually specifying a project name that does not include the slash works.
Could you please elaborate your query?
Based on your documentation here: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/docker-compose-v0?view=azure-pipelines#syntax The default value for the projectName parameter for the DockerCompose@0 task will be '$(Build.Repository.Name)'. For a GitHub repository this will resolve to 'owner/repo'. 'owner/repo' is an invalid docker compose project name in all instances. https://docs.docker.com/compose/project-name/#set-a-project-name
Project names must contain only lowercase letters, decimal digits, dashes, and underscores, and must begin with a lowercase letter or decimal digit. If the base name of the project directory or current directory violates this constraint, alternative mechanisms are available.
I'm having a similar issue. Here's some additional info:
ubuntu-latest
and DockerCompose@0 0.240.2
on dev.azure.com
ubuntu-latest
and DockerCompose@0 0.240.2
, same commit/
) Workaround for project name with uppercase letters: explicitly add projectName
in task inputs. Sample task definition:
- task: DockerCompose@0
displayName: Build docker compose services
inputs:
projectName: ${{ lower(variables['Build.Repository.Name']) }}
containerregistrytype: 'Azure Container Registry'
dockerComposeFile: 'docker-compose.yml'
action: 'Build services'
Unsuccessful run with system diagnostics (Original project name was CamelCase
, changed to Redacted
):
2024-06-20T21:20:18.1799389Z ##[debug]Evaluating condition for step: 'Build docker compose services' 2024-06-20T21:20:18.1800123Z ##[debug]Evaluating: SucceededNode() 2024-06-20T21:20:18.1800372Z ##[debug]Evaluating SucceededNode: 2024-06-20T21:20:18.1800756Z ##[debug]=> True 2024-06-20T21:20:18.1801002Z ##[debug]Result: True 2024-06-20T21:20:18.1801305Z ##[section]Starting: Build docker compose services 2024-06-20T21:20:18.1807981Z ============================================================================== 2024-06-20T21:20:18.1808132Z Task : Docker Compose 2024-06-20T21:20:18.1808228Z Description : Build, push or run multi-container Docker applications. Task can be used with Docker or Azure Container registry. 2024-06-20T21:20:18.1808393Z Version : 0.240.2 2024-06-20T21:20:18.1808462Z Author : Microsoft Corporation 2024-06-20T21:20:18.1808545Z Help : https://aka.ms/azpipes-docker-compose-tsg 2024-06-20T21:20:18.1808643Z ============================================================================== 2024-06-20T21:20:18.2278896Z ##[debug]Resource Utilization warnings is disabled, switch "DISABLE_RESOURCE_UTILIZATION_WARNINGS" variable to "true" if you want to enable it 2024-06-20T21:20:18.2727968Z ##[debug]Using node path: /home/vsts/agents/3.241.0/externals/node16/bin/node 2024-06-20T21:20:18.5606918Z ##[debug]agent.TempDirectory=/home/vsts/work/_temp 2024-06-20T21:20:18.5607393Z ##[debug]loading inputs and endpoints 2024-06-20T21:20:18.5608044Z ##[debug]loading INPUT_CONTAINERREGISTRYTYPE 2024-06-20T21:20:18.5622633Z ##[debug]loading INPUT_DOCKERCOMPOSEFILE 2024-06-20T21:20:18.5625273Z ##[debug]loading INPUT_PROJECTNAME 2024-06-20T21:20:18.5628442Z ##[debug]loading INPUT_QUALIFYIMAGENAMES 2024-06-20T21:20:18.5635212Z ##[debug]loading INPUT_ACTION 2024-06-20T21:20:18.5636000Z ##[debug]loading INPUT_INCLUDESOURCETAGS 2024-06-20T21:20:18.5636800Z ##[debug]loading INPUT_INCLUDELATESTTAG 2024-06-20T21:20:18.5638658Z ##[debug]loading INPUT_BUILDIMAGES 2024-06-20T21:20:18.5640145Z ##[debug]loading INPUT_DETACHED 2024-06-20T21:20:18.5642388Z ##[debug]loading INPUT_ABORTONCONTAINEREXIT 2024-06-20T21:20:18.5644933Z ##[debug]loading INPUT_IMAGEDIGESTCOMPOSEFILE 2024-06-20T21:20:18.5648403Z ##[debug]loading INPUT_REMOVEBUILDOPTIONS 2024-06-20T21:20:18.5650969Z ##[debug]loading INPUT_BASERESOLVEDIRECTORY 2024-06-20T21:20:18.5653653Z ##[debug]loading INPUT_OUTPUTDOCKERCOMPOSEFILE 2024-06-20T21:20:18.5654976Z ##[debug]loading INPUT_NOPIFNODOCKERCOMPOSEFILE 2024-06-20T21:20:18.5657206Z ##[debug]loading INPUT_REQUIREADDITIONALDOCKERCOMPOSEFILES 2024-06-20T21:20:18.5658576Z ##[debug]loading INPUT_CWD 2024-06-20T21:20:18.5660901Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION 2024-06-20T21:20:18.5661999Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION 2024-06-20T21:20:18.5664153Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN 2024-06-20T21:20:18.5668800Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN 2024-06-20T21:20:18.5670091Z ##[debug]loaded 21 2024-06-20T21:20:18.5674783Z ##[debug]Agent.ProxyUrl=undefined 2024-06-20T21:20:18.5675779Z ##[debug]Agent.CAInfo=undefined 2024-06-20T21:20:18.5676812Z ##[debug]Agent.ClientCert=undefined 2024-06-20T21:20:18.5677602Z ##[debug]Agent.SkipCertValidation=undefined 2024-06-20T21:20:18.6182756Z ##[debug]agent.proxyurl=undefined 2024-06-20T21:20:18.6183183Z ##[debug]VSTS_ARM_REST_IGNORE_SSL_ERRORS=undefined 2024-06-20T21:20:18.6183870Z ##[debug]AZURE_HTTP_USER_AGENT=VSTS_d0ee55bc-caac-469f-b347-2b3466c496d8_build_43_0 2024-06-20T21:20:18.7666538Z ##[debug]check path : /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json 2024-06-20T21:20:18.7675762Z ##[debug]adding resource file: /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json 2024-06-20T21:20:18.7676855Z ##[debug]system.culture=en-US 2024-06-20T21:20:18.7704130Z ##[debug]Agent.TempDirectory=/home/vsts/work/_temp 2024-06-20T21:20:18.7715301Z ##[debug]Resource file has already set to: /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-azure-arm-rest/module.json 2024-06-20T21:20:18.8097500Z ##[debug]check path : /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-docker-common/module.json 2024-06-20T21:20:18.8098172Z ##[debug]adding resource file: /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-docker-common/module.json 2024-06-20T21:20:18.8098606Z ##[debug]system.culture=en-US 2024-06-20T21:20:18.8102404Z ##[debug]Resource file has already set to: /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/node_modules/azure-pipelines-tasks-docker-common/module.json 2024-06-20T21:20:18.8110791Z ##[debug]check path : /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/task.json 2024-06-20T21:20:18.8111307Z ##[debug]adding resource file: /home/vsts/work/_tasks/DockerCompose_6975e2d1-96d3-4afc-8a41-498b5d34ea19/0.240.2/task.json 2024-06-20T21:20:18.8111723Z ##[debug]system.culture=en-US 2024-06-20T21:20:18.8120402Z ##[debug]cwd=/home/vsts/work/1/s 2024-06-20T21:20:18.8160586Z ##[debug]containerregistrytype=Azure Container Registry 2024-06-20T21:20:18.8160995Z ##[debug]azureSubscriptionEndpoint=undefined 2024-06-20T21:20:18.8161354Z ##[debug]azureContainerRegistry=undefined 2024-06-20T21:20:18.8161723Z ##[debug]Attempting to get endpoint authorization scheme... 2024-06-20T21:20:18.8162102Z ##[debug]Failed to get endpoint authorization scheme. 2024-06-20T21:20:18.8162506Z ##[debug]Attempting to get endpoint authorization scheme as an authorization parameter... 2024-06-20T21:20:18.8162981Z ##[debug]Failed to get endpoint authorization scheme as an authorization parameter. Will default authorization scheme to ServicePrincipal. 2024-06-20T21:20:18.8163436Z ##[debug]dockerComposeFile=/home/vsts/work/1/s/docker-compose.yml 2024-06-20T21:20:18.8163803Z ##[debug]nopIfNoDockerComposeFile=false 2024-06-20T21:20:18.8164169Z ##[debug]No pattern found in docker compose filepath parameter 2024-06-20T21:20:18.8196000Z ##[debug]which 'docker' 2024-06-20T21:20:18.8198895Z ##[debug]found: '/usr/bin/docker' 2024-06-20T21:20:18.8199306Z ##[debug]Feature flag USE_DOCKER_COMPOSE_V2_COMPATIBLE_MODE = true 2024-06-20T21:20:18.8205312Z ##[debug]dockerComposePath=undefined 2024-06-20T21:20:18.8209852Z ##[debug]which 'docker' 2024-06-20T21:20:18.8210168Z ##[debug]found: '/usr/bin/docker' 2024-06-20T21:20:18.8210532Z ##[debug]dockerComposeFile=/home/vsts/work/1/s/docker-compose.yml 2024-06-20T21:20:18.8210893Z ##[debug]cwd=/home/vsts/work/1/s 2024-06-20T21:20:18.8211249Z ##[debug]No pattern found in docker compose filepath parameter 2024-06-20T21:20:18.8211754Z ##[debug]additionalDockerComposeFiles=undefined 2024-06-20T21:20:18.8212229Z ##[debug]requireAdditionalDockerComposeFiles=false 2024-06-20T21:20:18.8212593Z ##[debug]projectName=Redacted 2024-06-20T21:20:18.8212929Z ##[debug]dockerHostEndpoint=undefined 2024-06-20T21:20:18.8213457Z ##[debug]dockerComposeFileArgs=undefined 2024-06-20T21:20:18.8213802Z ##[debug]which '/usr/bin/docker' 2024-06-20T21:20:18.8214134Z ##[debug]found: '/usr/bin/docker' 2024-06-20T21:20:18.8214471Z ##[debug]dockerComposePath=undefined 2024-06-20T21:20:18.8215120Z ##[debug]/usr/bin/docker arg: compose 2024-06-20T21:20:18.8215471Z ##[debug]/usr/bin/docker arg: --compatibility 2024-06-20T21:20:18.8215855Z ##[debug]/usr/bin/docker arg: ["-f","/home/vsts/work/1/s/docker-compose.yml"] 2024-06-20T21:20:18.8216215Z ##[debug]/usr/bin/docker arg: ["-p","Redacted"] 2024-06-20T21:20:18.8216563Z ##[debug]/usr/bin/docker arg: config 2024-06-20T21:20:18.8216900Z ##[debug]exec tool: /usr/bin/docker 2024-06-20T21:20:18.8217222Z ##[debug]arguments: 2024-06-20T21:20:18.8217531Z ##[debug] compose 2024-06-20T21:20:18.8217845Z ##[debug] --compatibility 2024-06-20T21:20:18.8218139Z ##[debug] -f 2024-06-20T21:20:18.8218609Z ##[debug] /home/vsts/work/1/s/docker-compose.yml 2024-06-20T21:20:18.8218940Z ##[debug] -p 2024-06-20T21:20:18.8219243Z ##[debug] Redacted 2024-06-20T21:20:18.8219551Z ##[debug] config 2024-06-20T21:20:19.3027707Z ##[debug]Agent environment resources - Disk: / Available 21380.00 MB out of 74244.00 MB, Memory: Used 723.00 MB out of 6921.00 MB, CPU: Usage 74.08% 2024-06-20T21:20:23.2158962Z ##[error]invalid project name "Redacted": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number 2024-06-20T21:20:23.2166661Z ##[debug]Processed: ##vso[task.issue type=error;]invalid project name "Redacted": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number 2024-06-20T21:20:23.2173529Z ##[debug]Exit code 15 received from tool '/usr/bin/docker' 2024-06-20T21:20:23.2180121Z ##[debug]STDIO streams have closed for tool '/usr/bin/docker' 2024-06-20T21:20:23.2203323Z ##[debug]task result: Failed 2024-06-20T21:20:23.2204083Z ##[error]The process '/usr/bin/docker' failed with exit code 15 2024-06-20T21:20:23.2204633Z ##[debug]Processed: ##vso[task.issue type=error;]The process '/usr/bin/docker' failed with exit code 15 2024-06-20T21:20:23.2205473Z ##[debug]Processed: ##vso[task.complete result=Failed;]The process '/usr/bin/docker' failed with exit code 15 2024-06-20T21:20:23.2286229Z ##[section]Finishing: Build docker compose services
Hello Everyone, For now we have disabled the FF DistributedTask.Agent.UseDockerComposeV2CompatibleMode for all the organizations. I hope you all will not be having any issues related to projectName parameters.
The root cause of this problem was introduced here: https://github.com/compose-spec/compose-go/pull/261
Docker compose now validates project names, defining a set of valid characters (of which /
is not one)
Add telemetry and migration warning for DockerComposeV0 #20064
We got it working during this previous days and started failing again today. Any suggestions?
The issue has returned for us as well. We are looking for guidance on how to fix.
azure-pipeline.yml
:
- task: DockerCompose@0
displayName: Install dependencies
inputs:
dockerComposeFile: .ci/pr/docker-compose.yml
action: Run a specific service
serviceName: ci
detached: false
workDir: /app
containerCommand: .ci/pr/install.bash
Error during pipelines execution:
##[warning]The project name "gitbub_org_name/gitbub_repo_name" must be a valid docker compose project name. Follow the link for more details: https://aka.ms/azdo-docker-compose-v1
##[error]invalid project name "gitbub_org_name/gitbub_repo_name": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number
##[error]The process '/usr/bin/docker' failed with exit code 15
Where gitbub_org_name/gitbub_repo_name
is your github repo.
azure-pipeline.yml
:- task: DockerCompose@0 displayName: Install dependencies inputs: dockerComposeFile: .ci/pr/docker-compose.yml action: Run a specific service serviceName: ci detached: false workDir: /app containerCommand: .ci/pr/install.bash
Error during pipelines execution:
##[warning]The project name "gitbub_org_name/gitbub_repo_name" must be a valid docker compose project name. Follow the link for more details: https://aka.ms/azdo-docker-compose-v1 ##[error]invalid project name "gitbub_org_name/gitbub_repo_name": must consist only of lowercase alphanumeric characters, hyphens, and underscores as well as start with a letter or number ##[error]The process '/usr/bin/docker' failed with exit code 15
Where
gitbub_org_name/gitbub_repo_name
is your github repo.
Could you try to add Project name in the task inputs and check if that works fine for you and make sure you follow the rules specified for Project name. Project names must contain only lowercase letters, decimal digits, dashes, and underscores, and must begin with a lowercase letter or decimal digit.
@v-schhabra thanks for looking into it. That worked:
- task: DockerCompose@0
displayName: Pull docker images
inputs:
projectName: test
...
Issue is mitigated so closing this out.
@v-schhabra I believe it is still an issue. I don't expect it to fail if I omit an optional configuration and the default value is not adhering to the standard
Are we saying we need to modify the 20+ projects that are using this to include this change on the pipeline? Any implications to that?
@gsouf, the DockerComposeV0 task uses docker-compose cli by default. The hotfix Docker Compose v0.242.10 has been released recently. Could you confirm the problem still occurs on this version of the DockerCompose task?
@bison92, you may continue to use DockerComposeV0 with your project names but there is an additional information about Docker Compose tasks changes
@ivanduplenskikh I don't think you need logs to reproduce the issue. The issue will occur, as explained above, when you run the pipelines from github without specifying the name. It will by default use the repository name that includes an invalid character (the slash "/")
@gsouf, as I mentioned earlier, the DockerComposeV0 task uses docker-compose CLI which doesn't impose strict constraints to a project name. I've tested this scenario with slash in a project name and it functioned as expected. The DockerComposeV0 task was successful, but the DockerComposeV1 task failed.
strategy:
matrix:
DockerCompose@0:
taskName: DockerCompose@0
DockerCompose@1:
taskName: DockerCompose@1
steps:
- powershell: |
@"
services:
alpine-service:
image: alpine:latest
"@ | Out-File -FilePath "./docker-compose.yml"
- task: $(taskName)
inputs:
containerregistrytype: 'Azure Container Registry'
dockerComposeFile: 'docker-compose.yml'
projectName: 'test/Test'
action: 'Build services'
New issue checklist
Task name
DockerCompose@0
Breaking task version
0.239.0
Last working task version
0.238.0
Regression Description
This functionality was broken in version 0.239.0, but is not observable unless the flag USE_DOCKER_COMPOSE_V2_COMPATIBLE_MODE is true.
The DockerCompose@0 task defaults projectName to '$(Build.Repository.Name)'. However, GitHub repositories have a forward slash (/) in their name, and the forward slash is an invalid property for projectName when calling docker compose. I'm not sure why this only just broke, but manually specifying a project name that does not include the slash works.
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
windows-2019
Relevant log output
Full task logs with system.debug enabled
UNSUCCESSFUL RUN
SUCCESSFUL RUN
Repro steps