search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.47k stars 2.6k forks source link

[REGRESSION]: AzurePowerShell Task - Error when calling Update-AzConfig in parallel #20226

Closed v-mBorisov closed 5 days ago

v-mBorisov commented 1 month ago

New issue checklist

Task name

No response

Breaking task version

242

Last working task version

241

Regression Description

Related to ADO ICM: https://portal.microsofticm.com/imp/v3/incidents/incident/526602860/summary

In release 242, a call to Update-AzConfig was added that call 'PSConfig.json'

https://github.com/microsoft/azure-pipelines-tasks/compare/v241...v242#diff-c520ded3c957b670319b7dbc38c894dd277ddd1127060bd29df898771fab340cL34

image

For this reason, customers is running parallel jobs on the same Windows agent under the same user account for each process receive an error.

_The process cannot access the file 'C:\Users\Service_TSDAZBLD.Azure\PSConfig.json' because it is being used by another process.

It might be worth considering using Get-AzConfig to check whether the Update-AzConfig call is necessary

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Windows 11

Relevant log output

2024-07-17T00:24:20.0142774Z VERBOSE: Leaving Initialize-AzModule.
2024-07-17T00:24:20.0233836Z ##[debug]Error record:
2024-07-17T00:24:20.2140488Z ##[debug]Update-AzConfig: E:\Agents\EPRIAGT001_4\_work\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.242.0\ps_modules\VstsAzureHelpers_\InitializeAzModuleFunctions.ps1:35
2024-07-17T00:24:20.2159863Z ##[debug]Line |
2024-07-17T00:24:20.2178804Z ##[debug]  35 |  …             Update-AzConfig -DisplayBreakingChangeWarning $false -App …
2024-07-17T00:24:20.2195423Z ##[debug]     |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2024-07-17T00:24:20.2224399Z ##[debug]     | The process cannot access the file 'C:\Users\Service_TSD_AZBLD\.Azure\PSConfig.json' because it is being used by
2024-07-17T00:24:20.2240856Z ##[debug]     | another process.
2024-07-17T00:24:20.2254450Z ##[debug]
2024-07-17T00:24:20.2284173Z ##[debug]Script stack trace:
2024-07-17T00:24:20.2337308Z ##[debug]at Initialize-AzModule, E:\Agents\EPRIAGT001_4\_work\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.242.0\ps_modules\VstsAzureHelpers_\InitializeAzModuleFunctions.ps1: line 35
2024-07-17T00:24:20.2354661Z ##[debug]at <ScriptBlock>, E:\Agents\EPRIAGT001_4\_work\_tasks\AzurePowerShell_72a1931b-effb-4d2e-8fd8-f8472a07cb62\5.242.0\CoreAz.ps1: line 31
2024-07-17T00:24:20.2391041Z ##[debug]at <ScriptBlock>, E:\Agents\EPRIAGT001_4\_work\_temp\59039b5c-a35e-4208-a2c2-6a8511eab55f.ps1: line 3
2024-07-17T00:24:20.2397155Z ##[debug]at <ScriptBlock>, <No file>: line 1
2024-07-17T00:24:20.2429434Z ##[debug]Exception:
2024-07-17T00:24:20.2505413Z ##[debug]System.IO.IOException: The process cannot access the file 'C:\Users\Service_TSD_AZBLD\.Azure\PSConfig.json' because it is being used by another process.

Pull Request:

https://github.com/microsoft/azure-pipelines-tasks/pull/20304

Full task logs with system.debug enabled

No response

Repro steps

No response

DanielMannInCycle commented 1 month ago

This also affects AzurePowerShell@4.

@v-mBorisov Do you have any idea when this issue will be fixed and published?

Until then, do you have any suggestions on a low-impact workaround for this issue? I'm working with a team that is currently affected by this.

v-mBorisov commented 1 month ago

@DanielMannInCycle Hi, sorry for delay, I'm planning to do this this week, but I'm not sure exactly. Unfortunately, there is a possibility that this will not be fixed completely, but we will only reduce the frequency of the error.

In the incident https://portal.microsofticm.com/imp/v3/incidents/incident/526602860/summary it is described that as a workaround we do not recommend using parallel launches of pipelines. Docs: https://learn.microsoft.com/en-us/powershell/module/az.accounts/update-azconfig?view=azps-12.1.0

image

DanielMannInCycle commented 1 month ago

@v-mBorisov But that's the thing -- we're not updating anything. The task is forcing it. This isn't behavior we want or need. Isn't a reasonable solution to add a flag for disabling the behavior entirely so we can just skip it?

v-mBorisov commented 1 month ago

@DanielMannInCycle I consulted with the team and found out that we cannot install the flag. The most convenient solution would be to update only when necessary. This will work for you.

I'm already working on a fix.

n-athan commented 4 weeks ago

Would it be possible to add -Scope Process to the call of Update-AZConfig so the recommendation from the Microsoft Docs is followed?

DanielMannInCycle commented 2 weeks ago

@v-mBorisov Any updates on this? When can we expect a fix to be released?

v-mBorisov commented 5 days ago

@DanielMannInCycle The fix was added a week ago and will be added to release 244.