search

microsoft / azure-pipelines-agent

Azure Pipelines Agent 🚀
MIT License
1.72k stars 865 forks source link

"The SSL connection could not be established" when configuring agent behind enterprise root cert #3599

Closed sphanley closed 1 year ago

sphanley commented 2 years ago

Agent Version and Platform

2.194.0, MacOS

Azure DevOps Type and Version

Azure DevOps Services (dev.azure.com)

What's not working?

I'm attempting to install the Azure DevOps agent behind an enterprise proxy which injects a root certificate. I've added that root certificate to the OS X keychain and marked it as trusted, and when I go to https://dev.azure.com in my browser, I see it successfully being served up as trusted using my organization's root cert. When I try to run config.sh to configure the agent, I get this message, though:

The SSL connection could not be established, see inner exception.

In the generated log, here's the error I see:

[2021-11-08 22:07:33Z ERR VisualStudioServices] GET request to https://dev.azure.com/progcloud/_apis/connectionData?connectOptions=1&lastChangeId=-1&lastChangeId64=-1 failed. System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception) (... more boilerplate stacktrace)

The closest thing to guidance I can find is the article on how to Run the agent with a self-signed certificate, which suggests using curl -v to validate that your SSL is working. When I run a curl on the URL in question, the response includes:

  • Server certificate:
  • subject: CN=dev.azure.com
  • start date: Nov 18 13:11:54 2020 GMT
  • expire date: Nov 18 13:11:54 2021 GMT
  • subjectAltName: host "dev.azure.com" matched cert's "dev.azure.com"
  • issuer: // my organization's issuer information
  • SSL certificate verify ok.

Is there something I'm missing? If my SSL trust chain is validating in the browser and via CURL, is there something else I need to do to cause the agent config utility to validate my cert?

anatolybolshakov commented 2 years ago

Hi @sphanley do you run curl command from the same host which you use for agent run? Please note that there is some difference between validation process made by agent and curl - so basically it can't be considered fully equal. Does it work with --sslskipcertvalidation flag enabled during agent configuration?

sphanley commented 2 years ago

Hello @anatolybolshakov! Thanks for the reply. Yes, I'm running the curl -v on the same host that I'm trying to register as an agent. And yes, the registration works if I use --skipsslcertvalidation, but when the agent is registered that way, I continue to get SSL validation failures when my agent tries to run restore a cache hit using the Cache@2 task, so I feel like there's a real issue certificate trust issue I still need to resolve.

Like you said, it seems clear that the agent is doing something different than curl or my web browsers when validating the certificates - I'm just trying to understand what's actually different.

sphanley commented 2 years ago

Any info you're able to provide about what the pipeline agent's cert validation process entails would be very helpful - we're still blocked by this issue.

tinylabspace commented 2 years ago

I'm unsure if this is true on Mac based agents but on Windows agents there are some pipeline tasks that still require a NODE_EXTRA_CA_CERTS environment variable to be set pointing at the certificate even when the server OS is functioning behind a corporate proxy that is doing packet inspection. The agent can register successfully but pipelines fail. In that scenario the agent is online and jobs trigger, but some tasks still fail with certificate validation errors without the variable being set.

sphanley commented 2 years ago

I just discovered that if I run export AZP_AGENT_USE_LEGACY_HTTP=true prior to running config.sh, then my SSL validation works as expected. @anatolybolshakov, if you or anyone more familiar with this codebase can help provide info on what may have changed about SSL validation between the legacy HTTP handler and the new default handler, it would be immensely helpful.

anatolybolshakov commented 2 years ago

Hi @sphanley with Net Core update (from v2.1 to v3.1) in v2.165.0 of the agent libcurl used by dotnet core in its HTTP client implementation on Linux has been replaced by SocketsHttpHandler. It is still possible to force the libcurl based implementation for 3.x Net Core by setting the UseSocketsHttpHandler option to false, which can be controlled in the agent via the UseLegacyHttpHandler configuration knob ('AZP_AGENT_USE_LEGACY_HTTP' environment variable). Unfortunately, SocketsHttpHandler has several known specific issues fixed in further Net Core versions - we are going to proceed on Net Core update, but until then I believe you can use this agent knob as a workaround. It would probably make sense to clarify in the repo https://github.com/dotnet/runtime - that this is the known issue already fixed with releases later than 3.1

dogukancagatay commented 2 years ago

I am experiencing the same error while trying to register my Deployment Group Agent (Azure Devops 2019) with --skipsslcertvalidation flag. Using Azure Pipelines Agent v2.196.1.

Using AZP_AGENT_USE_LEGACY_HTTP=true didn't work for me, but the error has changed:

System.Net.Http.CurlException: SSL peer certificate or SSH remote key was not OK
sphanley commented 2 years ago

If you're getting an error with --skipsslcertvalidation, then it sounds like you're running into something different than what I'm experiencing. In my case, --skipsslcertvalidation causes the agent registration to work successfully, but then I continue to get other SSL errors when trying to run certain pipeline tasks. But in my case, my certificate is valid and trusted by the MacOS machine, but for some reason won't pass the pipeline agent's cert validation.

Zapbbx commented 2 years ago

I have the same problem..

[2022-01-31 16:37:45Z INFO LocationServer] Unable to connect to https://dev.azure.com/xxxxx/. [2022-01-31 16:37:45Z ERR LocationServer] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) [2022-01-31 16:37:45Z INFO CommandSettings] Flag 'unattended': 'False' [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR (exception): [2022-01-31 16:37:45Z ERR Terminal] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.GetConnectionData(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.DetermineDeploymentType(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command) [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR: Failed to connect. Try again or ctrl-c to quit [2022-01-31 16:37:45Z INFO CommandSettings] Arg 'url': '' [2022-01-31 16:37:45Z INFO CommandSettings] Flag 'unattended': 'False' [2022-01-31 16:37:45Z INFO PromptManager] ReadValue [2022-01-31 16:37:45Z INFO Terminal] WRITE: Enter server URL > [2022-01-31 16:37:45Z INFO Terminal] READ LINE

dhlee90 commented 2 years ago

I have the same problem..

[2022-01-31 16:37:45Z INFO LocationServer] Unable to connect to https://dev.azure.com/xxxxx/. [2022-01-31 16:37:45Z ERR LocationServer] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) [2022-01-31 16:37:45Z INFO CommandSettings] Flag 'unattended': 'False' [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR (exception): [2022-01-31 16:37:45Z ERR Terminal] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.. ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host. --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token) at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request) at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken) at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.GetConnectionData(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.DetermineDeploymentType(String serverUrl, VssCredentials credentials, ILocationServer locationServer) at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command) [2022-01-31 16:37:45Z ERR Terminal] WRITE ERROR: Failed to connect. Try again or ctrl-c to quit [2022-01-31 16:37:45Z INFO CommandSettings] Arg 'url': '' [2022-01-31 16:37:45Z INFO CommandSettings] Flag 'unattended': 'False' [2022-01-31 16:37:45Z INFO PromptManager] ReadValue [2022-01-31 16:37:45Z INFO Terminal] WRITE: Enter server URL > [2022-01-31 16:37:45Z INFO Terminal] READ LINE

I'm having the same issue as well, started this morning..

mamift commented 2 years ago

Anyone getting this issue after 31 Jan 2022, I think it could be related to this:

https://devblogs.microsoft.com/visualstudio/azure-devops-requires-tls-1-2-on-all-connections-including-visual-studio

Going forward Azure DevOps will require TLS 1.2 for all HTTPS connections, including their web API and Git services

However, I am getting this error on a Windows Server 2016 machine, and I thought all server versions post 2012 R2 should be OK to automatically connect via TLS 1.2? At least according to this.

marvinvdongen commented 2 years ago

We are having the same issue since 31st of Jan 2022.

dhlee90 commented 2 years ago

Anyone getting this issue after 31 Jan 2022, I think it could be related to this:

https://devblogs.microsoft.com/visualstudio/azure-devops-requires-tls-1-2-on-all-connections-including-visual-studio

Going forward Azure DevOps will require TLS 1.2 for all HTTPS connections, including their web API and Git services

However, I am getting this error on a Windows Server 2016 machine, and I thought all server versions post 2012 R2 should be OK to automatically connect via TLS 1.2? At least according to this.

we were getting it on 2019 machines.. We were able to resolve one machine by adjusting some ciphers and tls registry settings.. however the next day feb 1, all the other machines started working again with no other intervention....

mamift commented 2 years ago

we were getting it on 2019 machines.. We were able to resolve one machine by adjusting some ciphers and tls registry settings.. however the next day feb 1, all the other machines started working again with no other intervention....

Very strange, but it appears to have resolved itself for my servers as well. I couldn't find any advisory for this in the Azure Outages RSS feed either (https://status.dev.azure.com/_rss).

StuiterSlurf commented 2 years ago

Azure deployment agent doesn't work since a couple of days anymore due to SSL issues.

I got the same problem. A couple of servers that can't connect to Azure DevOps anymore.

I found the following blog that tls 1.2 should be enabled.

https://devblogs.microsoft.com/visualstudio/azure-devops-requires-tls-1-2-on-all-connections-including-visual-studio/

I checked if we have the correct security protocols enabled. 

C:\azagent\A2> [System.Net.ServicePointManager]::SecurityProtocol
Ssl3, Tls, Tls12

I also followed the following to make sure TLS is enabled in the windows register. I added the values by hand because some didn't exists. 

Path                                                                                          Name                        Value
----                                                                                          ----                        -----
HKLM:\SOFTWARE\WOW6432Node\Microsoft\.NETFramework\v4.0.30319                                 SystemDefaultTlsVersions    1
HKLM:\SOFTWARE\WOW6432Node\Microsoft\.NETFramework\v4.0.30319                                 SchUseStrongCrypto          1
HKLM:\SOFTWARE\Microsoft\.NETFramework\v4.0.30319                                             SystemDefaultTlsVersions    1
HKLM:\SOFTWARE\Microsoft\.NETFramework\v4.0.30319                                             SchUseStrongCrypto          1
HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server    Enabled                     1
HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server    DisabledByDefault           0
HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client    Enabled                     1
HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client    DisabledByDefault           0

I posted the log below because I don't know what's going on. Does anyone else have any idea?

Log:

[2022-02-05 18:38:37Z INFO AgentProcess] Agent package win-x64.
[2022-02-05 18:38:37Z INFO AgentProcess] Running on Windows (X64).
[2022-02-05 18:38:37Z INFO AgentProcess] RuntimeInformation: Microsoft Windows 6.3.9600.
[2022-02-05 18:38:37Z INFO AgentProcess] Version: 2.198.2
[2022-02-05 18:38:37Z INFO AgentProcess] Commit: d1b85881abfe7b5e575af095daf0ee27e099b904
[2022-02-05 18:38:37Z INFO AgentProcess] Culture: nl-NL
[2022-02-05 18:38:37Z INFO AgentProcess] UI Culture: en-US
[2022-02-05 18:38:37Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:37Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:37Z INFO AgentProcess] Validating directory permissions for: 'C:\azagent\A2'
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Generation: '1'
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\1\PowerShellEngine', value name 'PowerShellVersion': '2.0'
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Unsupported version. Skipping.
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Generation: '3'
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\3\PowerShellEngine', value name 'PowerShellVersion': '4.0'
[2022-02-05 18:38:37Z INFO PowerShellExeUtil] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\PowerShell\3\PowerShellEngine', value name 'ApplicationBase': 'C:\Windows\System32\WindowsPowerShell\v1.0'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework', value name 'InstallRoot': 'C:\Windows\Microsoft.NET\Framework64\'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'SOFTWARE\Microsoft\NET Framework Setup\NDP' contains sub keys:
[2022-02-05 18:38:37Z INFO AgentProcess]  'CDF'
[2022-02-05 18:38:37Z INFO AgentProcess]  'v2.0.50727'
[2022-02-05 18:38:37Z INFO AgentProcess]  'v3.0'
[2022-02-05 18:38:37Z INFO AgentProcess]  'v3.5'
[2022-02-05 18:38:37Z INFO AgentProcess]  'v4'
[2022-02-05 18:38:37Z INFO AgentProcess]  'v4.0'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v2.0.50727', value name 'Version': '2.0.50727.4927'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v2.0.50727', value name 'Install': '1'
[2022-02-05 18:38:37Z INFO AgentProcess] Testing directory: 'C:\Windows\Microsoft.NET\Framework64\v2.0.50727'
[2022-02-05 18:38:37Z INFO AgentProcess] Found version: 2.0.50727
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v3.0', value name 'Version': '3.0.30729.4926'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v3.0', value name 'Install': '1'
[2022-02-05 18:38:37Z INFO AgentProcess] Testing directory: 'C:\Windows\Microsoft.NET\Framework64\v3.0'
[2022-02-05 18:38:37Z INFO AgentProcess] Found version: 3.0
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v3.5', value name 'Version': '3.5.30729.4926'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v3.5', value name 'Install': '1'
[2022-02-05 18:38:37Z INFO AgentProcess] Testing directory: 'C:\Windows\Microsoft.NET\Framework64\v3.5'
[2022-02-05 18:38:37Z INFO AgentProcess] Found version: 3.5
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4', value name 'Version' is null.
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4', value name '' is null.
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'SOFTWARE\Microsoft\NET Framework Setup\NDP\v4' contains sub keys:
[2022-02-05 18:38:37Z INFO AgentProcess]  'Client'
[2022-02-05 18:38:37Z INFO AgentProcess]  'Full'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Client', value name 'Version': '4.8.03761'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Client', value name 'Install': '1'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Client', value name 'InstallPath': 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Client', value name 'Release': '528049'
[2022-02-05 18:38:37Z INFO AgentProcess] Type is System.Int32
[2022-02-05 18:38:37Z INFO AgentProcess] Interpreted version: 4.7.0
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full', value name 'Version': '4.8.03761'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full', value name 'Install': '1'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full', value name 'InstallPath': 'C:\Windows\Microsoft.NET\Framework64\v4.0.30319\'
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full', value name 'Release': '528049'
[2022-02-05 18:38:37Z INFO AgentProcess] Type is System.Int32
[2022-02-05 18:38:37Z INFO AgentProcess] Interpreted version: 4.7.0
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4.0', value name 'Version' is null.
[2022-02-05 18:38:37Z INFO AgentProcess] Key name 'HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4.0', value name '': 'deprecated'
[2022-02-05 18:38:37Z INFO AgentProcess] Found 5 versions:
[2022-02-05 18:38:37Z INFO AgentProcess]  2.0.50727
[2022-02-05 18:38:37Z INFO AgentProcess]  3.0
[2022-02-05 18:38:37Z INFO AgentProcess]  3.5
[2022-02-05 18:38:37Z INFO AgentProcess]  4.7.0
[2022-02-05 18:38:37Z INFO AgentProcess]  4.7.0
[2022-02-05 18:38:37Z INFO AgentProcess] Testing for min NET Framework version: '4.5'
[2022-02-05 18:38:37Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:37Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO CommandSettings] Configure {
  "AcceptTeeEula": false,
  "AddDeploymentGroupTags": false,
  "AddEnvironmentVirtualMachineResourceTags": false,
  "AddMachineGroupTags": false,
  "AlwaysExtractTask": false,
  "Agent": "IIS18",
  "CollectionName": null,
  "DeploymentGroup": false,
  "DeploymentGroupName": null,
  "DeploymentGroupTags": null,
  "DeploymentPool": false,
  "DeploymentPoolName": null,
  "EnvironmentVMResource": true,
  "EnvironmentName": "Logic4-Next - Production",
  "EnvironmentVMResourceTags": null,
  "GitUseSChannel": false,
  "DisableLogUploads": false,
  "MachineGroup": false,
  "MachineGroupName": null,
  "MachineGroupTags": null,
  "MonitorSocketAddress": null,
  "NotificationPipeName": null,
  "NotificationSocketAddress": null,
  "NoRestart": false,
  "OverwriteAutoLogon": false,
  "Pool": null,
  "ProjectName": "Logic4Desktop",
  "ProxyPassword": null,
  "ProxyUserName": null,
  "ProxyUrl": null,
  "Replace": false,
  "RunAsAutoLogon": false,
  "RunAsService": true,
  "RunOnce": false,
  "PreventServiceStart": false,
  "SslCACert": null,
  "SslClientCert": null,
  "SslClientCertArchive": null,
  "SslClientCertKey": null,
  "SslClientCertPassword": null,
  "SslSkipCertValidation": false,
  "Url": "https://dev.azure.com/organization/",
  "WindowsLogonAccount": null,
  "WindowsLogonPassword": null,
  "Work": "_work",
  "Auth": "PAT",
  "LaunchBrowser": false,
  "Password": null,
  "Token": "***",
  "Unattended": false,
  "UserName": null,
  "Help": false,
  "Version": false
}
[2022-02-05 18:38:38Z INFO AgentProcess] Arguments parsed
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Proxy': 'C:\azagent\A2\.proxy'
[2022-02-05 18:38:38Z INFO VstsAgentWebProxy] No proxy setting found.
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Certificates': 'C:\azagent\A2\.certificates'
[2022-02-05 18:38:38Z INFO AgentCertificateManager] No certificate setting found.
[2022-02-05 18:38:38Z INFO Agent] ExecuteCommand
[2022-02-05 18:38:38Z INFO ConfigurationStore] currentAssemblyLocation: C:\azagent\A2\bin\Agent.Listener.dll
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO ConfigurationStore] binPath: C:\azagent\A2\bin
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO ConfigurationStore] RootFolder: C:\azagent\A2
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Agent': 'C:\azagent\A2\.agent'
[2022-02-05 18:38:38Z INFO ConfigurationStore] ConfigFilePath: C:\azagent\A2\.agent
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Credentials': 'C:\azagent\A2\.credentials'
[2022-02-05 18:38:38Z INFO ConfigurationStore] CredFilePath: C:\azagent\A2\.credentials
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Service': 'C:\azagent\A2\.service'
[2022-02-05 18:38:38Z INFO ConfigurationStore] ServiceConfigFilePath: C:\azagent\A2\.service
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Autologon': 'C:\azagent\A2\.autologon'
[2022-02-05 18:38:38Z INFO ConfigurationStore] AutoLogonSettingsFilePath: C:\azagent\A2\.autologon
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'Options': 'C:\azagent\A2\.options'
[2022-02-05 18:38:38Z INFO ConfigurationStore] RuntimeOptionsFilePath: C:\azagent\A2\.options
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO HostContext] Well known config file 'SetupInfo': 'C:\azagent\A2\.setup_info'
[2022-02-05 18:38:38Z INFO ConfigurationStore] SetupInfoFilePath: C:\azagent\A2\.setup_info
[2022-02-05 18:38:38Z INFO Terminal] WRITE LINE: 
  ___                      ______ _            _ _
 / _ \                     | ___ (_)          | (_)
/ /_\ \_____   _ _ __ ___  | |_/ /_ _ __   ___| |_ _ __   ___  ___
|  _  |_  / | | | '__/ _ \ |  __/| | '_ \ / _ \ | | '_ \ / _ \/ __|
| | | |/ /| |_| | | |  __/ | |   | | |_) |  __/ | | | | |  __/\__ \
\_| |_/___|\__,_|_|  \___| \_|   |_| .__/ \___|_|_|_| |_|\___||___/
                                   | |
        agent v2.198.2             |_|          (commit d1b8588)

[2022-02-05 18:38:38Z INFO ConfigurationManager] CheckAgentRootDirectorySecure
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Bin': 'C:\azagent\A2\bin'
[2022-02-05 18:38:38Z INFO HostContext] Well known directory 'Root': 'C:\azagent\A2'
[2022-02-05 18:38:38Z INFO ConfigurationManager] ConfigureAsync
[2022-02-05 18:38:38Z INFO ConfigurationStore] IsConfigured()
[2022-02-05 18:38:38Z INFO ConfigurationStore] IsConfigured: False
[2022-02-05 18:38:38Z INFO ConfigurationManager] Is configured: False
[2022-02-05 18:38:38Z INFO CommandSettings] Flag 'sslskipcertvalidation': 'False'
[2022-02-05 18:38:38Z INFO ConfigurationManager] Testing for min NET Framework version: '4.6'
[2022-02-05 18:38:38Z INFO CommandSettings] Flag 'deploymentgroup': 'False'
[2022-02-05 18:38:38Z INFO CommandSettings] Flag 'deploymentpool': 'False'
[2022-02-05 18:38:38Z INFO CommandSettings] Flag 'environment': 'True'
[2022-02-05 18:38:38Z INFO ExtensionManager] Getting extensions for interface: 'Microsoft.VisualStudio.Services.Agent.Listener.Configuration.IConfigurationProvider'
[2022-02-05 18:38:38Z INFO ExtensionManager] Creating instance: Microsoft.VisualStudio.Services.Agent.Listener.Configuration.BuildReleasesAgentConfigProvider, Agent.Listener
[2022-02-05 18:38:38Z INFO ExtensionManager] Creating instance: Microsoft.VisualStudio.Services.Agent.Listener.Configuration.DeploymentGroupAgentConfigProvider, Agent.Listener
[2022-02-05 18:38:38Z INFO ExtensionManager] Creating instance: Microsoft.VisualStudio.Services.Agent.Listener.Configuration.SharedDeploymentAgentConfigProvider, Agent.Listener
[2022-02-05 18:38:38Z INFO ExtensionManager] Creating instance: Microsoft.VisualStudio.Services.Agent.Listener.Configuration.EnvironmentVMResourceConfigProvider, Agent.Listener
[2022-02-05 18:38:38Z INFO Terminal] WRITE LINE: 
[2022-02-05 18:38:38Z INFO Terminal] WRITE LINE: >> Connect:
[2022-02-05 18:38:38Z INFO Terminal] WRITE LINE: 
[2022-02-05 18:38:38Z INFO CommandSettings] Arg 'url': 'https://dev.azure.com/organization/'
[2022-02-05 18:38:38Z INFO EnvironmentVMResourceConfigProvider] url - https://dev.azure.com/organization/
[2022-02-05 18:38:38Z INFO ConfigurationManager] GetCredentialProvider
[2022-02-05 18:38:38Z INFO CommandSettings] Arg 'auth': 'PAT'
[2022-02-05 18:38:38Z INFO ConfigurationManager] Creating credential for auth: PAT
[2022-02-05 18:38:38Z INFO CredentialManager] GetCredentialProvider
[2022-02-05 18:38:38Z INFO CredentialManager] Creating type PAT
[2022-02-05 18:38:38Z INFO CredentialManager] Creating credential type: PAT
[2022-02-05 18:38:38Z INFO PersonalAccessToken] EnsureCredential
[2022-02-05 18:38:38Z INFO CommandSettings] Arg 'token': '***'
[2022-02-05 18:38:38Z INFO PersonalAccessToken] GetVssCredentials
[2022-02-05 18:38:38Z INFO PersonalAccessToken] token retrieved: 52 chars
[2022-02-05 18:38:38Z INFO PersonalAccessToken] cred created
[2022-02-05 18:38:38Z INFO ConfigurationManager] cred retrieved
[2022-02-05 18:38:38Z INFO VisualStudioServices] Starting operation Location.GetConnectionData
[2022-02-05 18:38:38Z WARN VisualStudioServices] Attempt 1 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The operation will be retried in 10,8320613 seconds.
[2022-02-05 18:38:49Z WARN VisualStudioServices] Attempt 2 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The operation will be retried in 13,2434562 seconds.
[2022-02-05 18:39:03Z WARN VisualStudioServices] Attempt 3 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The operation will be retried in 15,8216932 seconds.
[2022-02-05 18:39:18Z ERR  VisualStudioServices] Attempt 4 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The maximum number of attempts has been reached.
[2022-02-05 18:39:18Z INFO VisualStudioServices] Finished operation Location.GetConnectionData
[2022-02-05 18:39:18Z INFO LocationServer] Unable to connect to https://dev.azure.com/organization/.
[2022-02-05 18:39:19Z ERR  LocationServer] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
 ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection)
[2022-02-05 18:39:19Z INFO CommandSettings] Flag 'unattended': 'False'
[2022-02-05 18:39:19Z ERR  Terminal] WRITE ERROR (exception):
[2022-02-05 18:39:19Z ERR  Terminal] System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
 ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.FixedSizeReader.ReadPacketAsync(Stream transport, AsyncProtocolRequest request)
   at System.Net.Security.SslStream.ThrowIfExceptional()
   at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
   at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result)
   at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
   at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__65_1(IAsyncResult iar)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.GetConnectionDataAsync(ConnectOptions connectOptions, Int64 lastChangeId, CancellationToken cancellationToken, Object userState)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.GetConnectionDataAsync(ConnectOptions connectOptions, Int32 lastChangeId, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.ConnectAsync(ConnectOptions connectOptions, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.LocationServer.ConnectAsync(VssConnection jobConnection)
   at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.GetConnectionData(String serverUrl, VssCredentials credentials, ILocationServer locationServer)
   at Microsoft.VisualStudio.Services.Agent.Util.ServerUtil.DetermineDeploymentType(String serverUrl, VssCredentials credentials, ILocationServer locationServer)
   at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command)
[2022-02-05 18:39:19Z ERR  Terminal] WRITE ERROR: Failed to connect.  Try again or ctrl-c to quit
[2022-02-05 18:39:59Z INFO CommandSettings] Arg 'url': 'https://dev.azure.com/organization/'
[2022-02-05 18:39:59Z INFO EnvironmentVMResourceConfigProvider] url - https://dev.azure.com/organization/
[2022-02-05 18:39:59Z INFO ConfigurationManager] GetCredentialProvider
[2022-02-05 18:39:59Z INFO CommandSettings] Arg 'auth': 'PAT'
[2022-02-05 18:39:59Z INFO ConfigurationManager] Creating credential for auth: PAT
[2022-02-05 18:39:59Z INFO CredentialManager] GetCredentialProvider
[2022-02-05 18:39:59Z INFO CredentialManager] Creating type PAT
[2022-02-05 18:39:59Z INFO CredentialManager] Creating credential type: PAT
[2022-02-05 18:39:59Z INFO PersonalAccessToken] EnsureCredential
[2022-02-05 18:39:59Z INFO CommandSettings] Arg 'token': '***'
[2022-02-05 18:39:59Z INFO PersonalAccessToken] GetVssCredentials
[2022-02-05 18:39:59Z INFO PersonalAccessToken] token retrieved: 52 chars
[2022-02-05 18:39:59Z INFO PersonalAccessToken] cred created
[2022-02-05 18:39:59Z INFO ConfigurationManager] cred retrieved
[2022-02-05 18:39:59Z INFO VisualStudioServices] Starting operation Location.GetConnectionData
[2022-02-05 18:39:59Z WARN VisualStudioServices] Attempt 1 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The operation will be retried in 10,8212117 seconds.
[2022-02-05 18:40:10Z WARN VisualStudioServices] Attempt 2 of GET request to https://dev.azure.com/organization/_apis/connectionData?connectOptions=1&lastChangeId=320929845&lastChangeId64=320929845 failed (Socket Error: ConnectionReset). The operation will be retried in 13,0277663 seconds.
[2022-02-05 18:40:11Z INFO Terminal] WRITE LINE: Exiting...
miguelperedo commented 2 years ago

Hi, I‘m having exact same error with https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/scale-set-agents?view=azure-devops Any news how to fix this? Thanks!

StuiterSlurf commented 2 years ago

@miguelperedo I also posted my problem at the developer community. I hope this helps for you as well :-)

https://developercommunity.visualstudio.com/t/Azure-DevOps-deployment-agent-cant-conn/1659981?space=21&entry=myfeedback

johnynfulleffect commented 2 years ago

Also having this issue when I upgraded to Ubuntu 22.04 in preparation for that release on 4/21/2022. I am running agents self hosted in docker, managed by kubernetes: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops

When i added the environment variable

AZP_AGENT_USE_LEGACY_HTTP=true

It works now.

speedpacer commented 2 years ago

I couldn't get this working on Ubuntu but was able to on Fedora 36 Server with just the following:

sudo dnf install dotnet-sdk-3.1 export AZP_AGENT_USE_LEGACY_HTTP=true

ruaan commented 1 year ago

Worked for me on Ubuntu 22.04 - export AZP_AGENT_USE_LEGACY_HTTP=true

ruaan commented 1 year ago

I would also recommend adding it to the runsvc.sh

!/bin/bash

convert SIGTERM signal to SIGINT

for more info on how to propagate SIGTERM to a child process see: http://veithen.github.io/2014/11/16/sigterm-propagation.html

trap 'kill -INT $PID' TERM INT

if [ -f ".path" ]; then

configure

export PATH=`cat .path`
echo ".path=${PATH}"

fi

insert anything to setup env when running as a service

export AZP_AGENT_USE_LEGACY_HTTP=true

dahol commented 1 year ago

I just discovered that if I run export AZP_AGENT_USE_LEGACY_HTTP=true prior to running config.sh, then my SSL validation works as expected. @anatolybolshakov, if you or anyone more familiar with this codebase can help provide info on what may have changed about SSL validation between the legacy HTTP handler and the new default handler, it would be immensely helpful.

Thank you very much, this works for me on a ubuntu 22.04 new install build agent

bbrougher-cerium commented 1 year ago

Running on Ubuntu 22.04.1 LTS I had to do the following to install the agent. Any one of the solutions above wouldn't work, I had to have a combination of the solutions:

mkdir ~/Downloads
cd ~/Downloads
wget https://vstsagentpackage.azureedge.net/agent/2.213.2/vsts-agent-linux-x64-2.213.2.tar.gz
mkdir ~/myagent && cd ~/myagent
tar zxvf ~/Downloads/vsts-agent-linux-x64-2.213.2.tar.gz
cd ~/Downloads
wget http://nz2.archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2_amd64.deb
sudo dpkg -i libssl1.1_1.1.1f-1ubuntu2_amd64.deb
cd ../myagent
export AZP_AGENT_USE_LEGACY_HTTP=true
./config.sh
sudo ./svc.sh install
sudo ./svc.sh start

Edit: I also had to add export AZP_AGENT_USE_LEGACY_HTTP=true to the top of the runsvc.sh file

playerjamesbattleground commented 1 year ago

@bbrougher-cerium can confirm we were on same boat and the env var use legacy http solved my issue too, thanks!

github-actions[bot] commented 1 year ago

This issue has had no activity in 180 days. Please comment if it is not actually stale

zeinali0 commented 10 months ago

If you're getting an error with --skipsslcertvalidation, then it sounds like you're running into something different than what I'm experiencing. In my case, --skipsslcertvalidation causes the agent registration to work successfully, but then I continue to get other SSL errors when trying to run certain pipeline tasks. But in my case, my certificate is valid and trusted by the MacOS machine, but for some reason won't pass the pipeline agent's cert validation.

I still have exactly same issue with vsts-agent-linux-x64-3.230.0 #4516

ajmalkhalil commented 6 months ago

I am facing the same issue, does anyone knows the fix for this? thanks

erstert3st commented 6 months ago

I am facing the same issue, does anyone knows the fix for this? thanks

same....

awaescher commented 5 months ago

We encountered the same issue and could not use our previously working macOS build agent anymore. I know that my approach is radical and please keep in mind that we have a 100% internal build server landscape serving an internal DevOps instance in our internal network.

Anyway ... we suddenly found this error when starting the agent:

[2024-04-19 08:15:51Z INFO Terminal] WRITE LINE: Connecting to the server.
[2024-04-19 08:15:51Z INFO MessageListener] Attempt to create session.
[2024-04-19 08:15:51Z INFO MessageListener] Connecting to the Agent Server...
[2024-04-19 08:15:51Z INFO AgentServer] Establish connection with 60 seconds timeout.
[2024-04-19 08:15:51Z INFO VisualStudioServices] Starting operation Location.GetConnectionData
[2024-04-19 08:15:51Z INFO AgentServer] Establish connection with 60 seconds timeout.
[2024-04-19 08:15:51Z INFO VisualStudioServices] Starting operation Location.GetConnectionData
[2024-04-19 08:15:51Z INFO AgentServer] Establish connection with 60 seconds timeout.
[2024-04-19 08:15:51Z INFO VisualStudioServices] Starting operation Location.GetConnectionData
[2024-04-19 08:15:51Z ERR  VisualStudioServices] GET request to https://XXXXXXXXXXXX/_apis/connectionData?connectOptions=1&lastChangeId=26&lastChangeId64=26 failed. System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.CurlException: SSL peer certificate or SSH remote key was not OK
   at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
   at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
[2024-04-19 08:15:51Z ERR  VisualStudioServices] GET request to https://XXXXXXXXXXXX/_apis/connectionData?connectOptions=1&lastChangeId=26&lastChangeId64=26 failed. System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.CurlException: SSL peer certificate or SSH remote key was not OK
   at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
   at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
[2024-04-19 08:15:51Z ERR  VisualStudioServices] GET request to https://XXXXXXXXXXXX/_apis/connectionData?connectOptions=1&lastChangeId=26&lastChangeId64=26 failed. System.Net.Http.HttpRequestException: An error occurred while sending the request.
 ---> System.Net.Http.CurlException: SSL peer certificate or SSH remote key was not OK
   at System.Net.Http.CurlHandler.ThrowIfCURLEError(CURLcode error)
   at System.Net.Http.CurlHandler.MultiAgent.FinishRequest(StrongToWeakReference`1 easyWrapper, CURLcode messageResult)
   --- End of inner exception stack trace ---
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)

Given the local environment and the fact that we wanted to run tests on our Mac build host temporarily, I used the hammer approach to make the build agent work again and that's ...

This will skip the certificate checks entirely and might be what you want have. Or it might not, depending on your environment.

image

⚠ Please make sure that if you follow these steps, your environment is isolated enough for this to be save! Only do this if you know what you are doing and be aware that it's on your own risk!

I decided to upload the patched DLL, it might only be compatible with vsts-agent-osx-x64-2.170.1 though.

Don't ever download DLL files from strangers on the internet

But in case you trust me (why should you?!), here's the patched version.

→ Microsoft.VisualStudio.Services.Common.dll.zip