search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.5k stars 2.61k forks source link

Docker tag/push images tasks EOL failures #7388

Closed mgreen closed 6 years ago

mgreen commented 6 years ago

Docker tag/push images tasks seem to fail on line endings. I created a simple 3-line DockerImages.txt file in my repo to use the Docker "Tag Images" task in my build. This task was working as recently as last Friday 06/01. I'm not sure what happened, but now this task is failing with errors that appear related to line endings.

Things I've tried:

Environment

Error logs

2018-06-06T04:48:23.2204894Z ##[section]Starting: Tag images
2018-06-06T04:48:23.2212357Z ==============================================================================
2018-06-06T04:48:23.2212694Z Task         : Docker
2018-06-06T04:48:23.2213188Z Description  : Build, tag, push, or run Docker images, or run a Docker command. Task can be used with Docker or Azure Container registry.
2018-06-06T04:48:23.2213555Z Version      : 0.3.12
2018-06-06T04:48:23.2213804Z Author       : Microsoft Corporation
2018-06-06T04:48:23.2214130Z Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=848006)
2018-06-06T04:48:23.2214477Z ==============================================================================
2018-06-06T04:48:23.6587186Z [command]"C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-dev ***.azurecr.io/mseg-cmc/rules-dev:latest
2018-06-06T04:48:24.2385726Z [command]"C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-prod ***.azurecr.io/mseg-cmc/rules-prod
2018-06-06T04:48:24.2388720Z :latest
2018-06-06T04:48:24.6636850Z Error parsing reference: "***.azurecr.io/mseg-cmc/rules-prod\r:latest" is not a valid repository/tag: invalid reference format
2018-06-06T04:48:24.6818495Z ##[error]C:\Program Files\Docker\Docker\Resources\bin\docker.exe failed with return code: 1
2018-06-06T04:48:24.6909871Z ##[section]Finishing: Tag images
vithati commented 6 years ago

"C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-dev .azurecr.io/mseg-cmc/rules-dev:latest here issue with . ***.azurecr.io is not vaild repo, so it throwing error. Can you try with proper name in repo and see issue exist or not.

mgreen commented 6 years ago

@vithati Sorry, i should have mentioned that I added the "*"s just to censor out the name of our ACR. It is actually the correct name in our logs.

vithati commented 6 years ago

Can you enable debug mode (set system.debug = true in build variables) and send the logs.

jikuma commented 6 years ago

There is back slash in the name of docker repository **.azurecr.io/mseg-cmc/rules-prod\r:latest. Are you sure its valid?. The regex for valid docker repository name is [a-z0-9]+(?:[._-][a-z0-9]+) . According to this backslash is not allowed. Removing \r in repository name should fix the issue.

vithati commented 6 years ago

\r not valid. But this line does not have "\r" in it. "C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-dev ***.azurecr.io/mseg-cmc/rules-dev:latest". Debug mode prints arguments. We can see arguments info.

jikuma commented 6 years ago

@vithati you are right. I was looking the error message.

Error parsing reference: "***.azurecr.io/mseg-cmc/rules-prod\r:latest" is not a valid repository/tag: invalid reference format.

This has \r in it.

mgreen commented 6 years ago

I will try to enable the debugging. Just fyi, \r is not in my repository or image names, I believe it's a bug in how the input file is generated/parsed.

mgreen commented 6 years ago

Here is the debug output. Again ***-ed out the name of the ACR.

Complete build can be found here.

2018-06-13T23:56:15.8404525Z ##[debug]Evaluating condition for step: 'Tag images'
2018-06-13T23:56:15.8405396Z ##[debug]Evaluating: succeeded()
2018-06-13T23:56:15.8405813Z ##[debug]Evaluating succeeded:
2018-06-13T23:56:15.8406310Z ##[debug]=> True
2018-06-13T23:56:15.8406808Z ##[debug]Result: True
2018-06-13T23:56:15.8409149Z ##[section]Starting: Tag images
2018-06-13T23:56:15.8414182Z ==============================================================================
2018-06-13T23:56:15.8414508Z Task         : Docker
2018-06-13T23:56:15.8414829Z Description  : Build, tag, push, or run Docker images, or run a Docker command. Task can be used with Docker or Azure Container registry.
2018-06-13T23:56:15.8415165Z Version      : 0.3.12
2018-06-13T23:56:15.8415412Z Author       : Microsoft Corporation
2018-06-13T23:56:15.8415693Z Help         : [More Information](https://go.microsoft.com/fwlink/?linkid=848006)
2018-06-13T23:56:15.8416027Z ==============================================================================
2018-06-13T23:56:16.1520423Z ##[debug]agent.workFolder=E:\agent_work
2018-06-13T23:56:16.1562438Z ##[debug]loading inputs and endpoints
2018-06-13T23:56:16.1577901Z ##[debug]loading ENDPOINT_AUTH_c5db4ea4-c979-4cea-9cf7-e2ccb722b848
2018-06-13T23:56:16.1591817Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_c5db4ea4-c979-4cea-9cf7-e2ccb722b848_SERVICEPRINCIPALID
2018-06-13T23:56:16.1595422Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_c5db4ea4-c979-4cea-9cf7-e2ccb722b848_SERVICEPRINCIPALKEY
2018-06-13T23:56:16.1601188Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_c5db4ea4-c979-4cea-9cf7-e2ccb722b848_TENANTID
2018-06-13T23:56:16.1603761Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2018-06-13T23:56:16.1607529Z ##[debug]loading ENDPOINT_AUTH_SCHEME_c5db4ea4-c979-4cea-9cf7-e2ccb722b848
2018-06-13T23:56:16.1611171Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2018-06-13T23:56:16.1613560Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2018-06-13T23:56:16.1617898Z ##[debug]loading INPUT_ACTION
2018-06-13T23:56:16.1622180Z ##[debug]loading INPUT_AZURECONTAINERREGISTRY
2018-06-13T23:56:16.1625128Z ##[debug]loading INPUT_AZURESUBSCRIPTIONENDPOINT
2018-06-13T23:56:16.1629679Z ##[debug]loading INPUT_CONTAINERREGISTRYTYPE
2018-06-13T23:56:16.1632528Z ##[debug]loading INPUT_CONTEXT
2018-06-13T23:56:16.1635237Z ##[debug]loading INPUT_CWD
2018-06-13T23:56:16.1645073Z ##[debug]loading INPUT_DEFAULTCONTEXT
2018-06-13T23:56:16.1645807Z ##[debug]loading INPUT_DETACHED
2018-06-13T23:56:16.1649625Z ##[debug]loading INPUT_DOCKERFILE
2018-06-13T23:56:16.1652525Z ##[debug]loading INPUT_ENFORCEDOCKERNAMINGCONVENTION
2018-06-13T23:56:16.1655032Z ##[debug]loading INPUT_IMAGEDIGESTFILE
2018-06-13T23:56:16.1657950Z ##[debug]loading INPUT_IMAGENAME
2018-06-13T23:56:16.1664100Z ##[debug]loading INPUT_IMAGENAMESPATH
2018-06-13T23:56:16.1669519Z ##[debug]loading INPUT_INCLUDELATESTTAG
2018-06-13T23:56:16.1682672Z ##[debug]loading INPUT_INCLUDESOURCETAGS
2018-06-13T23:56:16.1683490Z ##[debug]loading INPUT_QUALIFYIMAGENAME
2018-06-13T23:56:16.1684078Z ##[debug]loading INPUT_RESTARTPOLICY
2018-06-13T23:56:16.1684588Z ##[debug]loading SECRET_CONTAINER_AUTHENTICATIONTOKEN
2018-06-13T23:56:16.1685620Z ##[debug]loading SECRET_CONTAINER_PASSWORD
2018-06-13T23:56:16.1689888Z ##[debug]loading SECRET_CONTAINER_USERNAME
2018-06-13T23:56:16.1696905Z ##[debug]loaded 28
2018-06-13T23:56:16.2426263Z ##[debug]check path : E:\agent_work\_tasks\Docker_e28912f1-0114-4464-802a-a3a35437fd16\0.3.12\task.json
2018-06-13T23:56:16.2429420Z ##[debug]set resource file to: E:\agent_work\_tasks\Docker_e28912f1-0114-4464-802a-a3a35437fd16\0.3.12\task.json
2018-06-13T23:56:16.2430287Z ##[debug]system.culture=en-US
2018-06-13T23:56:16.2463994Z ##[debug]cwd=E:\agent_work\113\s
2018-06-13T23:56:16.2477603Z ##[debug]containerregistrytype=Azure Container Registry
2018-06-13T23:56:16.2479222Z ##[debug]azureSubscriptionEndpoint=c5db4ea4-c979-4cea-9cf7-e2ccb722b848
2018-06-13T23:56:16.2481642Z ##[debug]azureContainerRegistry={"loginServer":"***.azurecr.io", "id" : "/subscriptions/***/resourceGroups/container-rg/providers/Microsoft.ContainerRegistry/registries/***"}
2018-06-13T23:56:16.2486031Z ##[debug]c5db4ea4-c979-4cea-9cf7-e2ccb722b848 auth param serviceprincipalid = ***
2018-06-13T23:56:16.2487995Z ##[debug]c5db4ea4-c979-4cea-9cf7-e2ccb722b848 auth param serviceprincipalkey = ***
2018-06-13T23:56:16.2488912Z ##[debug]System.TeamFoundationCollectionUri=https://microsoft.visualstudio.com/
2018-06-13T23:56:16.2524038Z ##[debug]Processed: ##vso[task.setvariable variable=CONTAINER_USERNAME;issecret=true;]***
2018-06-13T23:56:16.2525533Z ##[debug]Processed: ##vso[task.setvariable variable=CONTAINER_PASSWORD;issecret=true;]***
2018-06-13T23:56:16.2538257Z ##[debug]Absolute path for pathSegments: E:\B\externals\git\cmd/docker = E:\B\externals\git\cmd\docker
2018-06-13T23:56:16.2542297Z ##[debug]Absolute path for pathSegments: C:\Program Files (x86)\Common Files\Oracle\Java\javapath/docker = C:\Program Files (x86)\Common Files\Oracle\Java\javapath\docker
2018-06-13T23:56:16.2553876Z ##[debug]Absolute path for pathSegments: C:\Program Files\Docker\Docker\Resources\bin/docker = C:\Program Files\Docker\Docker\Resources\bin\docker
2018-06-13T23:56:16.2559162Z ##[debug]check path : C:\Program Files\Docker\Docker\Resources\bin\docker.exe
2018-06-13T23:56:16.2560631Z ##[debug]docker=C:\Program Files\Docker\Docker\Resources\bin\docker.exe
2018-06-13T23:56:16.2561665Z ##[debug]dockerHostEndpoint=null
2018-06-13T23:56:16.2576873Z ##[debug]Processed: ##vso[task.setvariable variable=CONTAINER_AUTHENTICATIONTOKEN;issecret=true;]***
2018-06-13T23:56:16.5253695Z ##[debug]Synced the file content to the disk. The content is {"auths": { "***.azurecr.io": {"auth": "***", "email": "ServicePrincipal@AzureRM" } }, "HttpHeaders":{"X-Meta-Source-Client":"VSTS"} }.
2018-06-13T23:56:16.5255794Z ##[debug]action=Tag images
2018-06-13T23:56:16.7055435Z ##[debug]imageNamesPath=E:\agent_work\113\s\MSEG.CMC.Rules\DockerImages.txt
2018-06-13T23:56:16.7056283Z ##[debug]check path : E:\agent_work\113\s\MSEG.CMC.Rules\DockerImages.txt
2018-06-13T23:56:16.7060525Z ##[debug]enforceDockerNamingConvention=true
2018-06-13T23:56:16.7075409Z ##[debug]qualifyImageName=true
2018-06-13T23:56:16.7085822Z ##[debug]additionalImageTags=null
2018-06-13T23:56:16.7089831Z ##[debug]includeSourceTags=false
2018-06-13T23:56:16.7092516Z ##[debug]includeLatestTag=false
2018-06-13T23:56:16.7105070Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: tag
2018-06-13T23:56:16.7107341Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: mseg-cmc/rules-dev
2018-06-13T23:56:16.7108788Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: ***.azurecr.io/mseg-cmc/rules-dev:latest
2018-06-13T23:56:16.7109466Z ##[debug]Tagging image mseg-cmc/rules-dev with ***.azurecr.io/mseg-cmc/rules-dev:latest.
2018-06-13T23:56:16.7120881Z ##[debug]exec tool: C:\Program Files\Docker\Docker\Resources\bin\docker.exe
2018-06-13T23:56:16.7121627Z ##[debug]Arguments:
2018-06-13T23:56:16.7122135Z ##[debug]   tag
2018-06-13T23:56:16.7122648Z ##[debug]   mseg-cmc/rules-dev
2018-06-13T23:56:16.7123257Z ##[debug]   ***.azurecr.io/mseg-cmc/rules-dev:latest
2018-06-13T23:56:16.7135920Z [command]"C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-dev ***.azurecr.io/mseg-cmc/rules-dev:latest
2018-06-13T23:56:17.4497858Z ##[debug]rc:0
2018-06-13T23:56:17.4503781Z ##[debug]success:true
2018-06-13T23:56:17.4521544Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: tag
2018-06-13T23:56:17.4522583Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: mseg-cmc/rules-prod

2018-06-13T23:56:17.4523474Z ##[debug]C:\Program Files\Docker\Docker\Resources\bin\docker.exe arg: ***.azurecr.io/mseg-cmc/rules-prod
:latest
2018-06-13T23:56:17.4524308Z ##[debug]Tagging image mseg-cmc/rules-prod
 with ***.azurecr.io/mseg-cmc/rules-prod
:latest.
2018-06-13T23:56:17.4529007Z ##[debug]exec tool: C:\Program Files\Docker\Docker\Resources\bin\docker.exe
2018-06-13T23:56:17.4530025Z ##[debug]Arguments:
2018-06-13T23:56:17.4531975Z ##[debug]   tag
2018-06-13T23:56:17.4532670Z ##[debug]   mseg-cmc/rules-prod
2018-06-13T23:56:17.4533380Z ##[debug]   ***.azurecr.io/mseg-cmc/rules-prod
:latest
2018-06-13T23:56:17.4534121Z [command]"C:\Program Files\Docker\Docker\Resources\bin\docker.exe" tag mseg-cmc/rules-prod ***.azurecr.io/mseg-cmc/rules-prod
2018-06-13T23:56:17.4534676Z :latest
2018-06-13T23:56:17.9213815Z Error parsing reference: "***.azurecr.io/mseg-cmc/rules-prod\r:latest" is not a valid repository/tag: invalid reference format
2018-06-13T23:56:17.9259724Z ##[debug]rc:1
2018-06-13T23:56:17.9260621Z ##[debug]success:false
2018-06-13T23:56:17.9389150Z ##[debug]task result: Failed
2018-06-13T23:56:17.9391106Z ##[error]C:\Program Files\Docker\Docker\Resources\bin\docker.exe failed with return code: 1
2018-06-13T23:56:17.9392552Z ##[debug]Processed: ##vso[task.issue type=error;]C:\Program Files\Docker\Docker\Resources\bin\docker.exe failed with return code: 1
2018-06-13T23:56:17.9393438Z ##[debug]Processed: ##vso[task.complete result=Failed;]C:\Program Files\Docker\Docker\Resources\bin\docker.exe failed with return code: 1
2018-06-13T23:56:17.9461567Z ##[section]Finishing: Tag images
vithati commented 6 years ago

Thanks for reporting the issue. I found the bug. I am working on bug fix. Once bug fixed i will update this thread.

vithati commented 6 years ago

Hi, Issue fixed. PR https://github.com/Microsoft/vsts-tasks/pull/7473.