kishore-bhp
commented
1 year ago I am running a pipeline beyond proxy and getting timeout issue when pipeline is communicating with azure repositories. Attached logs for reference. logs_83659.zip
snipped of error occured :
2023-01-17T04:03:30.3425487Z ##[section]Async Command Start: Publish test results
2023-01-17T04:03:30.3426228Z ##[section]Async Command End: Publish test results
2023-01-17T04:03:30.3645104Z ##[error]The HTTP request timed out after 00:00:30.
2023-01-17T04:03:30.3675154Z ##[debug]System.TimeoutException: The HTTP request timed out after 00:00:30.
---> System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled.
---> System.Net.Sockets.SocketException (125): Operation canceled
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Http.HttpConnection.FillAsync()
at System.Net.Http.HttpConnection.ReadNextResponseHeaderLineAsync(Boolean foldedHeadersAllowed)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, HttpConnectionPool connectionPool, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.EstablishProxyTunnel(HttpRequestHeaders headers, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.LocationService.CreateDataProviderAsync(String location, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.LocationService.ResolveLocationDataAsync(Guid locationAreaIdentifier, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.LocationService.GetLocationDataAsync(Guid locationAreaIdentifier, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssConnection.GetClientInstanceAsync(Type managedType, Guid serviceIdentifier, CancellationToken cancellationToken, VssHttpRequestSettings settings, DelegatingHandler[] handlers) at Microsoft.VisualStudio.Services.WebApi.VssConnection.GetClientServiceImplAsync(Type requestedType, Guid serviceIdentifier, Func4 getInstanceAsync, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssConnection.GetClientAsync[T](Guid serviceIdentifier, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.TaskExtensions.SyncResult[T](Task`1 task)
at Microsoft.VisualStudio.Services.WebApi.VssConnection.GetClient[T](Guid serviceIdentifier)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.VssConnectionHelper.GetClient[T](Guid serviceIdentifier)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.PublishTestResultsCacheManager..ctor(IVssConnectionHelper vssConnectionHelper, TraceListener traceListener)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.TestRunPublisher..ctor(IVssConnectionHelper vssConnectionHelper, TraceListener traceListener, ITestResultsPublisher testResultsPublisher)
at Microsoft.TeamFoundation.TestClient.PublishTestResults.TestRunPublisher..ctor(VssConnection vssConnection, TraceListener traceListener)
at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.TestDataPublisher.InitializePublisher(IExecutionContext context, String projectName, VssConnection connection, String testRunner)
at Microsoft.VisualStudio.Services.Agent.Worker.TestResults.PublishTestResultsCommand.PublishTestRunDataAsync(String teamProject, TestRunContext testRunContext)
at Microsoft.VisualStudio.Services.Agent.Worker.AsyncCommandContext.WaitAsync()
at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
2023-01-17T04:03:30.3716163Z ##[section]Finishing: Build & Test
2023-01-17T04:03:31.9658839Z ##[debug]Evaluating condition for step: 'Deploy to Nexus'
2023-01-17T04:03:31.9662495Z ##[debug]Evaluating: SucceededNode()
2023-01-17T04:03:31.9663481Z ##[debug]Evaluating SucceededNode:
2023-01-17T04:03:31.9665513Z ##[debug]=> False
2023-01-17T04:03:31.9666576Z ##[debug]Result: False
2023-01-17T04:03:31.9710893Z ##[debug]Evaluating condition for step: 'Generate Nexus Credentials'
2023-01-17T04:03:31.9713310Z ##[debug]Evaluating: AlwaysNode()
2023-01-17T04:03:31.9714148Z ##[debug]Evaluating AlwaysNode:
2023-01-17T04:03:31.9716013Z ##[debug]=> True
2023-01-17T04:03:31.9716853Z ##[debug]Result: True
2023-01-17T04:03:31.9717673Z ##[section]Starting: Generate Nexus Credentials
2023-01-17T04:03:31.9729433Z ==============================================================================
2023-01-17T04:03:31.9729994Z Task : Maven Authenticate
2023-01-17T04:03:31.9730656Z Description : Provides credentials for Azure Artifacts feeds and external maven repositories
2023-01-17T04:03:31.9731439Z Version : 0.213.0
2023-01-17T04:03:31.9732004Z Author : Microsoft Corporation
2023-01-17T04:03:31.9732573Z Help : https://go.microsoft.com/fwlink/?linkid=2099968
2023-01-17T04:03:31.9732913Z ==============================================================================
2023-01-17T04:03:33.1440520Z ##[debug]Using node path: /opt/azure-agent/externals/node16/bin/node
2023-01-17T04:03:33.2433435Z ##[debug]agent.TempDirectory=/opt/azure-agent/_work/_temp
2023-01-17T04:03:33.2444274Z ##[debug]loading inputs and endpoints
2023-01-17T04:03:33.2445739Z ##[debug]loading INPUT_MAVENSERVICECONNECTIONS
2023-01-17T04:03:33.2456539Z ##[debug]loading ENDPOINT_AUTH_51c5c5df-7073-46b1-8b43-c350f238b2c5
2023-01-17T04:03:33.2458465Z ##[debug]loading ENDPOINT_AUTH_SCHEME_51c5c5df-7073-46b1-8b43-c350f238b2c5
2023-01-17T04:03:33.2459905Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_51c5c5df-7073-46b1-8b43-c350f238b2c5_USERNAME
2023-01-17T04:03:33.2461325Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_51c5c5df-7073-46b1-8b43-c350f238b2c5_PASSWORD
2023-01-17T04:03:33.2462337Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2023-01-17T04:03:33.2463825Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2023-01-17T04:03:33.2466100Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2023-01-17T04:03:33.2470575Z ##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
2023-01-17T04:03:33.2472709Z ##[debug]loading SECRET_VADDITIONALPROPERTIES
2023-01-17T04:03:33.2475268Z ##[debug]loading SECRET_SOLACEPASSWORD
2023-01-17T04:03:33.2477601Z ##[debug]loading SECRET_ADDITIONALPROPERTIES
2023-01-17T04:03:33.2479948Z ##[debug]loading SECRET_ESBMESSAGINGXA2PASSWORD
2023-01-17T04:03:33.2482185Z ##[debug]loading SECRET_ESBMESSAGINGNONXA1PASSWORD
2023-01-17T04:03:33.2483955Z ##[debug]loading SECRET_ESBMESSAGINGNONXA2PASSWORD
2023-01-17T04:03:33.2485227Z ##[debug]loading SECRET_ESBMESSAGINGPASSWORD
2023-01-17T04:03:33.2486476Z ##[debug]loading SECRET_ESBMESSAGINGXA1PASSWORD
2023-01-17T04:03:33.2488073Z ##[debug]loading VSTS_TASKVARIABLE_BACKUPUSERM2SETTINGSFILEPATH
2023-01-17T04:03:33.2488919Z ##[debug]loading VSTS_TASKVARIABLE_USERM2SETTINGSXMLPATH
2023-01-17T04:03:33.2489694Z ##[debug]loaded 19
2023-01-17T04:03:33.2493915Z ##[debug]Agent.ProxyUrl=http://proxy-per.pergtm.bhp.com:8081
2023-01-17T04:03:33.2494902Z ##[debug]Agent.ProxyUsername=undefined
2023-01-17T04:03:33.2495679Z ##[debug]Agent.ProxyPassword=undefined
2023-01-17T04:03:33.2496799Z ##[debug]Agent.ProxyBypassList=["127\.0\.0\.1","10\.0\.0\.0/8","localhost","\.bhp\.com","\.bhpbilliton\.net","127\.0\.0\.1","10\.0\.0\.0/8","localhost","\.bhp\.com","\.bhpbilliton\.net"]
2023-01-17T04:03:33.2497863Z ##[debug]expose agent proxy configuration.
2023-01-17T04:03:33.2498635Z ##[debug]Agent.CAInfo=undefined
2023-01-17T04:03:33.2499396Z ##[debug]Agent.ClientCert=undefined
2023-01-17T04:03:33.2500158Z ##[debug]Agent.SkipCertValidation=undefined
2023-01-17T04:03:33.2501301Z ##[debug]check path : /opt/azure-agent/_work/_tasks/MavenAuthenticate_d4b964f9-ea90-41bb-9526-29589628ad90/0.213.0/task.json
2023-01-17T04:03:33.2502801Z ##[debug]adding resource file: /opt/azure-agent/_work/_tasks/MavenAuthenticate_d4b964f9-ea90-41bb-9526-29589628ad90/0.213.0/task.json
2023-01-17T04:03:33.2503944Z ##[debug]system.culture=en-US
2023-01-17T04:03:33.2511494Z ##[debug]Agent.Version=2.213.2
2023-01-17T04:03:33.2514454Z ##[debug]task variable: userM2SettingsXmlPath=/home/azauser/.m2/settings.xml
2023-01-17T04:03:33.2515343Z ##[debug]Agent.Version=2.213.2
2023-01-17T04:03:33.2516868Z ##[debug]task variable: backupUserM2SettingsFilePath=/home/azauser/.m2/_settings.xml
2023-01-17T04:03:33.2518293Z ##[debug]rm -rf /home/azauser/.m2/settings.xml
2023-01-17T04:03:33.2519337Z ##[debug]removing file
2023-01-17T04:03:33.2521167Z ##[debug]Deleted user m2 settings.xml file: /home/azauser/.m2/settings.xml
2023-01-17T04:03:33.2553209Z ##[debug]Restored old user m2 settings.xml file: /home/azauser/.m2/_settings.xml
2023-01-17T04:03:33.2603352Z ##[section]Finishing: Generate Nexus Credentials
2023-01-17T04:03:34.2354963Z ##[debug]Evaluating condition for step: 'Checkout entsub-rst-rlgexec-driverlog@feature_rtpdev to s'
2023-01-17T04:03:34.2356869Z ##[debug]Evaluating: AlwaysNode()
2023-01-17T04:03:34.2357346Z ##[debug]Evaluating AlwaysNode:
2023-01-17T04:03:34.2357987Z ##[debug]=> True
2023-01-17T04:03:34.2358495Z ##[debug]Result: True
2023-01-17T04:03:34.2359090Z ##[section]Starting: Checkout entsub-rst-rlgexec-driverlog@feature_rtpdev to s
2023-01-17T04:03:34.2365263Z ==============================================================================
2023-01-17T04:03:34.2365581Z Task : Get sources
2023-01-17T04:03:34.2365904Z Description : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.
2023-01-17T04:03:34.2366212Z Version : 1.0.0
2023-01-17T04:03:34.2366539Z Author : Microsoft
2023-01-17T04:03:34.2366887Z Help : More Information
2023-01-17T04:03:34.2367291Z ==============================================================================
2023-01-17T04:03:35.8255418Z Cleaning any cached credential from repository: entsub-rst-rlgexec-driverlog (Git)
2023-01-17T04:03:35.8272501Z ##[debug]Repository url=https://BHP-INT@dev.azure.com/BHP-INT/AU-WEST-WAIO/_git/entsub-rst-rlgexec-driverlog
2023-01-17T04:03:35.8276308Z ##[debug]targetPath=/opt/azure-agent/_work/9/s
2023-01-17T04:03:35.8324422Z ##[debug]Cleaned up /opt/azure-agent/externals/tee and /opt/azure-agent/_work/_temp/tee_temp_dir
2023-01-17T04:03:35.8385642Z ##[section]Finishing: Checkout entsub-rst-rlgexec-driverlog@feature_rtpdev to s
2023-01-17T04:04:39.8979634Z ##[section]Starting: Finalize Job
2023-01-17T04:04:39.9015628Z Cleaning up task key
2023-01-17T04:04:39.9017860Z Start cleaning up orphan processes.
2023-01-17T04:04:39.9529967Z ##[section]Finishing: Finalize Job
2023-01-17T04:04:39.9623872Z ##[debug]Starting diagnostic file upload.
2023-01-17T04:04:39.9624344Z ##[debug]Setting up diagnostic log folders.
2023-01-17T04:04:39.9627611Z ##[debug]Creating diagnostic log files folder.
2023-01-17T04:04:39.9628364Z ##[debug]Creating diagnostic log environment file.
2023-01-17T04:04:40.0427996Z ##[debug]Creating capabilities file.
2023-01-17T04:04:40.0481340Z ##[debug]Copying 1 worker diag logs from /opt/azure-agent/_diag.
2023-01-17T04:04:40.0528534Z ##[debug]Copying 1 agent diag logs from /opt/azure-agent/_diag.
2023-01-17T04:04:40.0537114Z ##[debug]Dumping of waagent.conf file
2023-01-17T04:04:40.0540338Z ##[debug]waagent.conf file wasn't found. Dumping was not done.
2023-01-17T04:04:40.0540634Z ##[debug]Dumping cloud-init logs.
2023-01-17T04:04:40.0568663Z ##[debug]Cloud-init isn't found on current machine.
2023-01-17T04:04:40.0569167Z ##[debug]Cloud-init logs were not found.
2023-01-17T04:04:40.0569422Z ##[debug]Dumping cloud-init logs is ended.
2023-01-17T04:04:40.0570077Z ##[debug]The platform is not based on Debian - skipping debsums check.
2023-01-17T04:04:40.0570389Z ##[debug]Starting dumping Agent Azure VM extension logs.
2023-01-17T04:04:40.0576654Z ##[debug]Path to Agent Azure VM Linux extension logs does not exist.
2023-01-17T04:04:40.0577072Z ##[debug](directory "/var/log/azure/Microsoft.VisualStudio.Services.TeamServicesAgentLinux" not found)
2023-01-17T04:04:40.0577458Z ##[debug]Agent Azure VM extension logs not found. Skipping.
2023-01-17T04:04:40.0577807Z ##[debug]Zipping diagnostic files.
2023-01-17T04:04:40.0629250Z ##[debug]Uploading diagnostic metadata file.
2023-01-17T04:04:40.0676945Z ##[debug]Diagnostic file upload complete.
2023-01-17T04:04:40.0719224Z ##[section]Finishing: MuleLibraryDeployment
ivanduplenskikh
github-actions[bot]
Having issue with YAML?
Please log an issue at Azure-Pipelines-YAML. Over there we discuss YAML templates, samples for Azure Pipelines, and designs for upcoming YAML features. Also a place for the community to share best practices, ideas, and so on. File suggestions and issues here if they're specific to YAML pipelines.
Having issue with Tasks?
Log an issue at Azure-Pipelines-Tasks. It contains all of the in-box tasks we ship with Azure-Pipelines/VSTS/TFS. If you're having issues with tasks in Build/Release jobs (e.g. unreasonable task failure) please log an issue there.
Having issue with software on Hosted Agent?
Log an issue at Hosted Agent Image Repository. It contains the VM image used in the Azure Pipelines Hosted Agent Pool. If you're having Build/Release failures that seems like they are related to software installed on the Hosted Agent (e.g. the
dotnetSDK is missing or the Azure SDK is not on the latest version) please log an issue there.Having generic issue with Azure-Pipelines/VSTS/TFS?
Please report it on Developer Community
Have you tried troubleshooting?
Troubleshooting doc
Agent Version and Platform
Version of your agent? 2.144.0/2.144.1/...
OS of the machine running the agent? OSX/Windows/Linux/...
Azure DevOps Type and Version
dev.azure.com (formerly visualstudio.com) or on-premises TFS/Azure DevOps Server?
If on-premises, which release? 2015.0, 2017.1, 2019 RC2, etc.
If dev.azure.com, what is your organization name? https://dev.azure.com/{organization} or https://{organization}.visualstudio.com
What's not working?
Please include error messages and screenshots.
Agent and Worker's Diagnostic Logs
Logs are located in the agent's
_diagfolder. The agent logs are prefixed withAgent_and the worker logs are prefixed withWorker_. All sensitive information should already be masked out, but please double-check before pasting here.