Timeout for task PublishADFTask@1 while executing using self hosted windows agent pool

[anujmittalcdw]

While running the task PublishADFTask@1, we are facing timeout issues. We have already verified the json templates. In each different run, it fails for different object while creating linked service. Some time the same linked service is deployed successfully in previous run.

Please find below error details for the same.

[debug]+ ~~~~

[debug] + CategoryInfo : CloseError: (😃 [New-AzResource], Exception

[debug] + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceCmdlet

[debug]

[debug]Script stack trace:

[debug]at Deploy-AdfObjectOnly, D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObjectOnly.ps1: line 129

[debug]at Deploy-AdfObject, D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObject.ps1: line 43

[debug]at , D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObject.ps1: line 37

[debug]at Deploy-AdfObject, D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObject.ps1: line 24

[debug]at , D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObject.ps1: line 37

[debug]at Deploy-AdfObject, D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\private\Deploy-AdfObject.ps1: line 24

[debug]at , D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\public\Publish-AdfV2FromJson.ps1: line 215

[debug]at Publish-AdfV2FromJson, D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\ps_modules\azure.datafactory.tools\public\Publish-AdfV2FromJson.ps1: line 214

[debug]at , D:\agents\agent2_work_tasks\PublishADFTask_1af843b5-35a0-411f-9a18-9eb7a59fb8b8\1.14.1173\PublishADF.ps1: line 130

[debug]at , : line 1

[debug]at , : line 22

[debug]at , : line 18

[debug]at , : line 1

[debug]Exception:

[debug]System.Exception: Operation failed because a request timed out.

[debug] at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.HandleException(ExceptionDispatchInfo capturedException)

[debug] at Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.ResourceManagerCmdletBase.ExecuteCmdlet()

[debug] at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()

[error]Operation failed because a request timed out.

[debug]Processed: ##vso[task.logissue type=error]Operation failed because a request timed out.

[debug]Processed: ##vso[task.complete result=Failed]

[NowinskiK]

It fails on this step:

New-AzResource `
            -ResourceType $resType `
            -ResourceGroupName $resourceGroupName `
            -Name "$resName" `
            -ApiVersion "2018-06-01" `
            -Properties $json `
            -IsFullObject -Force | Out-Null

which means the DevOps Agent does NOT have access to the API. Check the network configuration.

[anujmittalcdw]

Hi NowinskiK

This issue we are facing is intermittent. Sometimes its falling sometimes its passing. If it would have been network issue then it will be failing each time.

For Example In attached Snip1: It got failed for ICM SQL Ingestion.json In attached Snip2: In the next run, it is successfully deployed without making any changes.

and each time we run it fails for different trigger Json file and sometimes whole pipeline gets deployed successfully without any issues.

[NowinskiK]

I understand. That clearly means that there is nothing to do with the tools. It's either problem with network (Yes, network stability is variable) or DevOps Agent. The code remains unchanged, there is no moving/changing parts. Try to run the same deployment from your local PC if you are able.