Open pyramid-scheme-ceo opened 2 weeks ago
I started running into this today as well but it's on a self hosted agent. Last successful run was yesterday, few runs today mostly on the same pipelines, others are working
We are also facing the same issue any update on this or any workaround
Hello, started running into the same issue since yesterday as well
we are also facing same issue , what is quick fix?
Also in my company we are facing this issue now more and more. What helps to a certain degree is to use the RetryOnErrorCount feature on those steps.
Hello guys, The only workaround I found for now is to download my artifact with the download step.
As you can't set a target path with the download step, I added an extra step after to copy (CopyFiles@2) my downloaded artifact to the target path I was using with DownloadPipelineArtifact@2. Hope it helps !
This issue has been affecting some our production builds for over a week now, is there no system in place to revert the active version on these tasks from your end, no A/B testing? Pinning to 2.197.0 also doesn't work on our end, leaving us only with the (no offense) convoluted workaround posted by @sebastienchoisy.
Please look into this guys...
One more thing that I have realized is that we got the error often from files within the workspace folder.
And in the pipeline that produced the errors we were not using the workspace - clean: all feature defined in the pipeline yml file.
Once we added the workspace clean action there the errors did no more happen.
But of course I do not know if cleaning the workspace is an option for you. For us it was.
@MTomBosch are you on self hosted agents? According to the docs, the workspace on MS hosted ones is always clean https://learn.microsoft.com/en-us/azure/devops/pipelines/process/phases?view=azure-devops&tabs=yaml#workspace
Yes, we are on self-hosted agents.
New issue checklist
Task name
DownloadPipelineArtifact@2
Task version
2.198.0
Issue Description
Our pipeline has started failing to Download artifacts today for only one of the three jobs we run. All jobs run Ubuntu 20.04 and are running the same version of the task (2.198.0) yet one download fails and the other succeeds.
The jobs are:
Jobs 2 and 3 run in parallel.
The download artifact step always works in job 2 and always fails in job 3 with an error similar to this:
Note that the error still happens when hitting re-running failed jobs (meaning that job 3 is running by itself and no longer in parallel with job 2).
The step is defined in yaml as
Idk if it's relevant but it's weird to me that the logs seems to say that it downloads the artifact three times (may just be automatic retries, or it could genuinely be trying to download three times at once leading to the exception).
Environment type (Please select at least one enviroment where you face this issue)
Azure DevOps Server type
dev.azure.com (formerly visualstudio.com)
Azure DevOps Server Version (if applicable)
No response
Operation system
Ubuntu 20.04
Relevant log output
2024-10-18T01:01:48.1228874Z ##[section]Starting: Download Artifact: web-library 2024-10-18T01:01:48.1239540Z ============================================================================== 2024-10-18T01:01:48.1239729Z Task : Download Pipeline Artifacts 2024-10-18T01:01:48.1239833Z Description : Download build and pipeline artifacts 2024-10-18T01:01:48.1239951Z Version : 2.198.0 2024-10-18T01:01:48.1240059Z Author : Microsoft Corporation 2024-10-18T01:01:48.1240162Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/download-pipeline-artifact 2024-10-18T01:01:48.1240347Z ============================================================================== 2024-10-18T01:01:48.5003431Z Download from the specified build: #29147 2024-10-18T01:01:48.5012319Z Download artifact to: /home/vsts/work/1/s/frontend 2024-10-18T01:01:49.0092512Z Using default max parallelism. 2024-10-18T01:01:49.3305795Z Using default max parallelism. 2024-10-18T01:01:49.3315173Z Max dedup parallelism: 192 2024-10-18T01:01:49.3315468Z DomainId: 0 2024-10-18T01:01:49.5066287Z ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:01:49.5220111Z Hashtype: Dedup1024K 2024-10-18T01:01:49.5468277Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:01:49.5486666Z Minimatch patterns: [] 2024-10-18T01:01:52.7566041Z Filtered 212943 files from the Minimatch filters supplied. 2024-10-18T01:01:52.7937816Z Downloaded 0.0 MB out of 1,491.6 MB (0%). 2024-10-18T01:01:57.7209800Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:01:57.7211262Z Minimatch patterns: [] 2024-10-18T01:01:57.8023489Z Downloaded 18.2 MB out of 1,491.6 MB (1%). 2024-10-18T01:02:00.4086215Z Filtered 212943 files from the Minimatch filters supplied. 2024-10-18T01:02:00.4356045Z Downloaded 0.0 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:02.8219624Z Downloaded 0.4 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:05.3630500Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:02:05.3631217Z Minimatch patterns: [] 2024-10-18T01:02:05.4342391Z Downloaded 7.5 MB out of 1,491.6 MB (1%). 2024-10-18T01:02:07.8217646Z Downloaded 45.4 MB out of 1,491.6 MB (3%). 2024-10-18T01:02:08.8305651Z Filtered 212943 files from the Minimatch filters supplied. 2024-10-18T01:02:08.8600550Z Downloaded 0.0 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:10.4782365Z Downloaded 2.4 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:12.8315497Z Downloaded 17.7 MB out of 1,491.6 MB (1%). 2024-10-18T01:02:13.8563909Z Downloaded 17.7 MB out of 1,491.6 MB (1%). 2024-10-18T01:02:15.4806993Z Downloaded 17.7 MB out of 1,491.6 MB (1%). 2024-10-18T01:02:16.3305178Z DedupManifestArtifactClient will correlate http requests with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:02:16.3305905Z Minimatch patterns: [] 2024-10-18T01:02:17.8401104Z Downloaded 55.6 MB out of 1,491.6 MB (4%). 2024-10-18T01:02:18.8531063Z Filtered 212943 files from the Minimatch filters supplied. 2024-10-18T01:02:18.8587161Z Downloaded 0.0 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:18.8789027Z Downloaded 0.0 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:20.4834981Z Downloaded 1.4 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:22.8395907Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:23.8592996Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:23.8800270Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:25.4828261Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:27.8352090Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:28.8554510Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:28.8757898Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:30.4855085Z Downloaded 4.1 MB out of 1,491.6 MB (0%). 2024-10-18T01:02:31.3926950Z ApplicationInsightsTelemetrySender correlated 2 events with X-TFS-Session b8bd3eea-be52-4afd-bf27-b689d1efef97 2024-10-18T01:02:31.3978588Z ##[error]The process cannot access the file '/home/vsts/work/1/s/frontend/node_modules/js-yaml/bin/js-yaml.js' because it is being used by another process. 2024-10-18T01:02:31.5266723Z ##[section]Finishing: Download Artifact: web-library
Full task logs with system.debug enabled
(Fyi this log only shows the first instance of the exception otherwise it goes past the max limit of this field)
Repro steps