Failed to download task 'Download' when using YAML Pipeline

ghost commented 4 years ago

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: DownloadPipelineArtifact

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

Server - Azure Pipelines or TFS on-premises? Azure Pipelines
- If using TFS on-premises, provide the version: N/A
- If using Azure Pipelines, provide the account name, team project name, build definition name/build number: Org: DemandCreationProgram Project: DC Program Team Pipeline: data-aggregator Id: 2126
Agent - Hosted or Private:
- If using Hosted agent, provide agent queue name: windows-2019

Issue Description

During the job initialization, I get an error message that it failed to download the 'Download' task. I am trying to use the download keyword in my YAML pipeline file.

Task logs

2020-05-11T00:35:07.3809773Z ##[section]Starting: Initialize job
2020-05-11T00:35:07.3811067Z Agent name: 'Hosted Agent'
2020-05-11T00:35:07.3811409Z Agent machine name: 'fv-az677'
2020-05-11T00:35:07.3811619Z Current agent version: '2.168.2'
2020-05-11T00:35:07.3849602Z ##[group]Operating System
2020-05-11T00:35:07.3849854Z Microsoft Windows Server 2019
2020-05-11T00:35:07.3850024Z 10.0.17763
2020-05-11T00:35:07.3850149Z Datacenter
2020-05-11T00:35:07.3850279Z ##[endgroup]
2020-05-11T00:35:07.3850429Z ##[group]Virtual Environment
2020-05-11T00:35:07.3850620Z Environment: windows-2019
2020-05-11T00:35:07.3850783Z Version: 20200430.2
2020-05-11T00:35:07.3851066Z Included Software: https://github.com/actions/virtual-environments/blob/win19/20200430.2/images/win/Windows2019-Readme.md
2020-05-11T00:35:07.3851459Z ##[endgroup]
2020-05-11T00:35:07.3852316Z Current image version: '20200430.2'
2020-05-11T00:35:07.3858221Z Agent running as: 'VssAdministrator'
2020-05-11T00:35:07.3916335Z ##[debug]Triggering repository: data-aggregator. repository type: Git
2020-05-11T00:35:07.3922829Z Prepare build directory.
2020-05-11T00:35:07.4212393Z ##[debug]Creating build directory: 'd:\a\1'
2020-05-11T00:35:07.4214950Z ##[debug]Delete existing artifacts directory: 'd:\a\1\a'
2020-05-11T00:35:07.4217444Z ##[debug]Creating artifacts directory: 'd:\a\1\a'
2020-05-11T00:35:07.4219424Z ##[debug]Delete existing test results directory: 'd:\a\1\TestResults'
2020-05-11T00:35:07.4220107Z ##[debug]Creating test results directory: 'd:\a\1\TestResults'
2020-05-11T00:35:07.4225297Z ##[debug]Creating binaries directory: 'd:\a\1\b'
2020-05-11T00:35:07.4227498Z ##[debug]Creating source directory: 'd:\a\1\s'
2020-05-11T00:35:07.4270001Z Set build variables.
2020-05-11T00:35:07.4309002Z Download all required tasks.
2020-05-11T00:35:07.4438032Z Downloading task: DownloadPipelineArtifact (1.2.4)
2020-05-11T00:35:07.6583114Z ##[debug]Task 'DownloadPipelineArtifact' has been downloaded into 'd:\a\_tasks\DownloadPipelineArtifact_61f2a582-95ae-4948-b34d-a1b3c4f6a737\1.2.4'.
2020-05-11T00:35:08.3627274Z Downloading task: Download (1.0.0)
2020-05-11T00:35:08.4441489Z ##[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
2020-05-11T00:35:08.4454917Z ##[warning]Back off 27.339 seconds before retry.
2020-05-11T00:35:36.4269703Z ##[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
2020-05-11T00:35:36.4274058Z ##[warning]Back off 18.883 seconds before retry.
2020-05-11T00:35:55.3577240Z ##[error]No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
2020-05-11T00:35:55.3584302Z ##[debug]Microsoft.TeamFoundation.DistributedTask.WebApi.TaskDefinitionNotFoundException: No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.TeamFoundation.DistributedTask.WebApi.TaskAgentHttpClient.GetTaskContentZipAsync(Guid taskId, TaskVersion version, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskManager.DownloadAsync(IExecutionContext executionContext, TaskStepDefinitionReference task)
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskManager.DownloadAsync(IExecutionContext executionContext, IEnumerable`1 steps)
   at Microsoft.VisualStudio.Services.Agent.Worker.JobExtension.InitializeJob(IExecutionContext jobContext, AgentJobRequestMessage message)
2020-05-11T00:35:55.3597591Z ##[section]Finishing: Initialize job

## Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

Starting: Initialize job
Agent name: 'Hosted Agent'
Agent machine name: 'fv-az674'
Current agent version: '2.168.2'
Operating System
Virtual Environment
Current image version: '20200430.2'
Agent running as: 'VssAdministrator'
Prepare build directory.
Set build variables.
Download all required tasks.
Downloading task: DownloadPipelineArtifact (1.2.4)
Downloading task: Download (1.0.0)
##[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
##[warning]Back off 12.591 seconds before retry.
##[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
##[warning]Back off 24.588 seconds before retry.
##[error]No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.
Finishing: Initialize job

joerianto83 commented 4 years ago

Hi guys,

got the same issue here, in Azure Devops, on MS hosted agent. But what i see is that it happends "randomly". i have just run the pipeline again , 2 jobs which contains the download steps, seems it is successfull now, but after(now) i tried another pipeline with a download task/start this same job again , it fails again.

Error: Starting: Initialize job Agent name: 'Hosted Agent' Agent machine name: 'WIN-E0GLM87UFV4' Current agent version: '2.166.4' Current image version: '20200430.2' Agent running as: 'VssAdministrator' Prepare build directory. Set build variables. Download all required tasks. Downloading task: Download (1.0.0)

[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.

[warning]Back off 24.134 seconds before retry.

[warning]Failed to download task 'Download'. Error No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.

[warning]Back off 11.536 seconds before retry.

[error]No task definition found matching ID 30f35852-3f7e-4c0c-9a88-e127b4f97211 and version 1.0.0. You must register the task definition before uploading the package.

Finishing: Initialize job

arek-avanade commented 4 years ago

I encountered that as well...

ericsciple commented 4 years ago

Same here. The download keyword in YAML pipelines doesnt work.

Sourceless commented 4 years ago

I encountered this today, switching to the DownloadPipelineArtifact task fixed this for me.

ghost commented 4 years ago

I encountered this today, switching to the DownloadPipelineArtifact task fixed this for me.

Yes, that worked for me as well. The documentation, however, says that I should be able to use the download keyword as an alias for that task. https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema#download

I think the keyword syntax is easier to read and follow as compared to the task which is why I opened the bug (aside from the fact that the functionality is broken). I just wish we could get a response from Microsoft on this.

AArnott commented 4 years ago

I was just hit by this. But it was on a pipeline I was modifying in another area. I use the download alias in all my pipelines and this morning is the first time I've hit it. Re-running that failed job alone several times did not alleviate it. I wonder if it is the version that it's looking for that's wrong. I notice it's looking for 1.0.0, but on successful runs I see that it actually uses 0.167.2.

AArnott commented 4 years ago

Dude: I think I found the cause. When I have something like this:

- download: CI
  artifact: some-artifact

It fails unless CI is in fact a defined resource in the pipeline. In my case, I had moved this step from one pipeline to another and forgot to change CI to current, so it was pointing at an invalid resource.

My guess is that Azure Pipelines wants to download the version of the task that matches the version of the tool used to upload it, so perhaps it tries to look up CI to get the version of download to install on the agent. Since CI does not exist, instead of failing with that error, it just defaults to version 1.0.0 which does not exist. When I switch to current as the pipeline source, it succeeds at installing 0.167.2.

So I think this is a case of a misleading error message when download is given the wrong inputs. It should have been this, in my case:

- download: current
  artifact: some-artifact

alexgurman commented 3 years ago

I am seeing the error when I use variables to specify download specs, using hard coded values works. Reported to Support team as a bug.

trajano commented 3 years ago

It needs a better error message that's for sure @AArnott comment at least let me realize what went wrong.

KevinKelchen commented 3 years ago

I am seeing the error when I use variables to specify download specs, using hard coded values works. Reported to Support team as a bug.

I likewise experienced this issue using a variable for the pipeline resource identifier like:

- download: '$(MyVariableName)'

I have not had any issues hardcoding a value of current.

However, when I switched from using a variable to a parameter, then it worked fine:

- download: '${{ parameters.MyParameterName }}'

Not sure if it made a difference, but my usage of the download task was within a YAML template and as part of a deployment job.

Perhaps the value for download is evaluated earlier in the pipeline's execution (compile-time, when the Tasks are downloaded, etc.). I worked around a similar limitation with trying to use a variable in a deployment job's environment value.

anatolybolshakov commented 3 years ago

Hi everyone, could you please open a ticket on https://developercommunity.visualstudio.com/search?space=21 to get right eyes on it? This error message comes from sever - since this parameter should be constant in runtime to work it properly at the moment, but error message could be definitely improved for this case. Let me close this one here since it's not related to the pipeline task.

AArnott commented 3 years ago

I have filed it here: https://developercommunity.visualstudio.com/t/download-task-fails-with-taskdefinitionnotfoundexc/1469872?from=email

Everyone on this thread please help vote it up.

Marty1703 commented 1 year ago

You need the variable at compile time, thus before runtime. Therefore another syntax must be used, see variable syntax.

I had the same issue and resolved it using the above documentation, thus ${{ variables.var }} over $(var).

microsoft / azure-pipelines-tasks