Self-Hosted Agent "Error has occurred while checking if system supports .NET 6: System.Net.Http.HttpRequestException"

[KocamanFaruk]

## Agent Version and Platform 2.217.2

## OS of the machine running the agent? Server 2019

## Azure DevOps Type and Version Version Azure DevOps Server 2020 Update 1.2

## What's not working? Our agent machine does not have internet access so getting this error message on Worker.log. Before upgrade did not have this issue. Inıtialize Job takes a long time to start because of this error.

## Agent and Worker's Diagnostic Logs [2023-02-13 13:56:16Z ERR JobRunner] Error has occurred while checking if system supports .NET 6: System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Agent.Sdk.PlatformUtil.GetNet6SupportedSystems() in D:\a\1\s\src\Agent.Sdk\Util\PlatformUtil.cs:line 320 at Agent.Sdk.PlatformUtil.DoesSystemPersistsInNet6Whitelist() in D:\a\1\s\src\Agent.Sdk\Util\PlatformUtil.cs:line 353 at Microsoft.VisualStudio.Services.Agent.Worker.JobRunner.RunAsync(AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken) in D:\a\1\s\src\Agent.Worker\JobRunner.cs:line 117

[max-zaytsev]

@KocamanFaruk thank you for reporting. We'll take a look at this

[KocamanFaruk]

We temporarily solve the problem, allowed only below url in agent machine;

https://raw.githubusercontent.com/microsoft/azure-pipelines-agent/master/src/Agent.Listener/net6.json

[nikolai-fra]

We are running into the same issue.

The "Worker_*" log under /agent/_diag shows

[2023-02-21 15:32:29Z INFO HostContext] Well known directory 'Bin': '/agent/bin'
[2023-02-21 15:32:29Z INFO HostContext] Well known directory 'Root': '/agent'
[2023-02-21 15:32:44Z INFO JobServerQueue] Stop aggressive process web console line queue.
[2023-02-21 15:34:09Z ERR  JobRunner] Error has occurred while checking if system supports .NET 6: System.Threading.Tasks.TaskCanceledException: The operation was canceled.
at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at Agent.Sdk.PlatformUtil.GetNet6SupportedSystems() in /home/vsts/work/1/s/src/Agent.Sdk/Util/PlatformUtil.cs:line 320
at Agent.Sdk.PlatformUtil.DoesSystemPersistsInNet6Whitelist() in /home/vsts/work/1/s/src/Agent.Sdk/Util/PlatformUtil.cs:line 353
at Microsoft.VisualStudio.Services.Agent.Worker.JobRunner.RunAsync(AgentJobRequestMessage message, CancellationToken jobRequestCancellationToken) in /home/vsts/work/1/s/src/Agent.Worker/J
obRunner.cs:line 117
[2023-02-21 15:34:09Z INFO JobRunner] Starting the job execution context.

Please notice the 1min28s gap between the log lines before the ERROR line. During this time the job is shown as accepted in Azure DevOps UI but it takes always ~around this time till the job really started

So it seems the job context is started after the HTTP request times out and the ERROR message form above is displayed

Some info to our environment:

Agent Version and Platform

Self-Hosted Azure Linux Virtual Machine Scale Set in an air-gapped environment. We have allowlisted the URLs from the docs here and here The extension version for "Microsoft.Azure.DevOps.Pipelines.Agent" is "1.23"

OS of the machine running the agent?

UbuntuLTS

Azure DevOps Type and Version

Azure DevOps SaaS

[ismayilov-ismayil]

Related PR: https://github.com/microsoft/azure-pipelines-agent/pull/4167

[KarenTazayan]

It seems that I have the same problem. Please look at this topic.

Every job stuck one the following lines for about 1 minute 20 sec and continue:

[2023-04-05 12:32:35Z INFO JobServerQueue] Stop aggressive process web console line queue.
[2023-04-05 12:34:00Z INFO HostContext] Well known directory 'Bin': '/home/u1/myagent/bin'

[KarenTazayan]

Any news on when it will be fixed?

[kirill-ivlev]

@KarenTazayan, @KocamanFaruk now this functionality moved to a separate non-blocking thread, which will not affect the agent performance. Closing this issue, please let us know if it is still an issue for you.

[anilakuzum]

@KarenTazayan, @KocamanFaruk I'm also experiencing the same issue, how did you resolve it?

[KarenTazayan]

@anilakuzum you should just update an agent software to the latest version.

[anilakuzum]

@KarenTazayan the version is up to date, I am using version 2.217.2

[KarenTazayan]

@anilakuzum the latest version at this point is 3.224.1.

[anilakuzum]

thank you , problem solved