search

Azure / azure-powershell

Microsoft Azure PowerShell
Other
4.21k stars 3.81k forks source link

Various errors such as "An error occurred while sending the request" #22562

Open chaoscreater opened 1 year ago

chaoscreater commented 1 year ago

Description

I'm on Windows 11 22H2 build 22621.2134. I'm using Powershell 7.3.6 and have the latest AZ module installed. I do not have any old AZ* module versions installed.

I'm running a very simple script that loops through each of my subscriptions, which fetches the app service, front door, application gateway and key vaults and store them in a array variable. Sometimes when running the script, it will throw the error An error occurred while sending the request and various other errors. Even running Get-AZSubscription or Set-AZContext will give me errors.

I've tried both with and without the following set at the top of my script:

[Net.ServicePointManager]::SecurityProtocol = 'Tls', 'Tls11','Tls12'

I've also checked the registry and there aren't any explicit configurations for the protocols or ciphers: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols

I've used IIS Crypto and selected best practices and that set the TLS 1.0~1.3 protocols to enabled and included the ciphers as well. However, none of this made any difference.

I've also checked this path against a fresh new Windows 11 install and the configuration is the same: Computer\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010003

I've also deleted the folder: %localappdata%\.IdentityService

and %USERPROFILE%\.azure

and then re-authenticated from scratch using Connect-AZAccount.

I have 2 routers at home. The second one is bridged to the primary one and I'm using a powerline adapter to bridge them. So it goes from router A -> ethernet cable -> powerline adapter A -> electrical wiring in the wall -> powerline adapter B -> ethernet cable -> router B.

My laptop is connected to router B. I've noticed that whenever I'm connected to router B, I get those errors in Powershell easily. But if I'm connected to router A or my phone's hotspot, then I don't get these errors at all. That leads me to believe perhaps the issue is the powerline or the router. I've swapped out router B and the ethernet cables and even the powerline adapters, but the issue is still there. And in case anyone asks, I cannot run ethernet cables from router A to my room, for reasons that I won't get into.

The string thing is that on another laptop running Windows 10 connected to router B, it works completely fine without any errors, I've run many many tests and the results are very consistent. It's also using the same Powershell version and same module version. Only difference is that it's Win10 and not Win11.

I've also done a Wireshark trace and can share the trace file if required. I'm not a network engineer and don't really know how to analyze it other than the basics, but it seems like after the ACK, my client sent a RESET to the target (Azure)?

image

I've also tried this in Powershell 5 and the result is the same.

Issue script & Debug output

Comment too long, see Pastebin:
https://pastebin.com/t68ECKKm

Environment data

Name                           Value
----                           -----
PSVersion                      7.3.6
PSEdition                      Core
GitCommitId                    7.3.6
OS                             Microsoft Windows 10.0.22621
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Module versions

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Script     2.12.5                Az.Accounts                         {Add-AzEnvironment, Clear-AzConfig, Clear-AzContext, Clear-AzDefault…}
Script     4.0.2                 Az.ApiManagement                    {Add-AzApiManagementApiToGateway, Add-AzApiManagementApiToProduct, Add-AzApiManagementProductToGroup, Add-AzApiManagementRegion…}
Script     1.10.0                Az.FrontDoor                        {Disable-AzFrontDoorCustomDomainHttps, Enable-AzFrontDoorCustomDomainHttps, Get-AzFrontDoor, Get-AzFrontDoorFrontendEndpoint…}
Script     4.10.1                Az.KeyVault                         {Add-AzKeyVaultCertificate, Add-AzKeyVaultCertificateContact, Add-AzKeyVaultKey, Add-AzKeyVaultManagedStorageAccount…}
Script     6.1.1                 Az.Network                          {Add-AzApplicationGatewayAuthenticationCertificate, Add-AzApplicationGatewayBackendAddressPool, Add-AzApplicationGatewayBackendHttpSetting…
Script     3.1.0                 Az.Websites                         {Add-AzWebAppAccessRestrictionRule, Add-AzWebAppTrafficRouting, Edit-AzWebAppBackupConfiguration, Get-AzAppServiceEnvironment…}
Script     2.0.0                 AzAppGWCert                         Get-AzAppGWCert

Error output

Message        : An error occurred while sending the request.
StackTrace     :    at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
                    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryAfterDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass15_0.<<SendAsync>b__0>d.MoveNext()
                 --- End of stack trace from previous location ---
                    at Microsoft.Rest.RetryDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Models.RPRegistrationDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Models.RPRegistrationDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.ClaimsChallengeHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Factories.CancelRetryHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts,
                 CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
                    at Microsoft.Azure.Management.ResourceManager.Version2021_01_01.SubscriptionsOperations.GetWithHttpMessagesAsync(String subscriptionId, Dictionary`2 customHeaders, CancellationToken
                 cancellationToken)
                    at Microsoft.Azure.Management.ResourceManager.Version2021_01_01.SubscriptionsOperationsExtensions.GetAsync(ISubscriptionsOperations operations, String subscriptionId, CancellationToken
                 cancellationToken)
                    at Microsoft.Azure.Management.ResourceManager.Version2021_01_01.SubscriptionsOperationsExtensions.Get(ISubscriptionsOperations operations, String subscriptionId)
                    at Microsoft.Azure.Commands.ResourceManager.Common.Utilities.Version2021_01_01.SubscriptionClientWrapper.GetSubscriptionById(String subscriptionId, IAccessToken accessToken, IAzureAccount
                 account, IAzureEnvironment environment)
                    at Microsoft.Azure.Commands.Profile.Models.SubscriptionClientProxy.<>c__DisplayClass6_0.<GetSubscriptionById>b__0(ISubscriptionClientWrapper client)
                    at Microsoft.Azure.Commands.Profile.Models.SubscriptionClientProxy.<>c.<.ctor>b__2_0(Action`1 subscriptionClientAction, LoggerWriter warning)
                    at Microsoft.Azure.Commands.Profile.Models.SubscriptionClientProxy.GetSubscriptionById(String subscriptionId, IAccessToken accessToken, IAzureAccount account, IAzureEnvironment
                 environment)
                    at Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient.TryGetSubscriptionById(String tenantId, String subscriptionId)
                    at Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient.SetCurrentContext(String subscriptionNameOrId, String tenantId, String name)
                    at Microsoft.Azure.Commands.Profile.SetAzureRMContextCommand.<>c__DisplayClass37_0.<ExecuteCmdlet>b__2(AzureRmProfile profile, RMProfileClient client, String name)
                    at Microsoft.Azure.Commands.Profile.Common.AzureContextModificationCmdlet.ModifyContext(Action`2 contextAction)
                    at Microsoft.Azure.Commands.Profile.SetAzureRMContextCommand.SetContextWithOverwritePrompt(Action`3 setContextAction)
                    at Microsoft.Azure.Commands.Profile.SetAzureRMContextCommand.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : System.Net.Http.HttpRequestException
InvocationInfo : {Set-AzContext}
Line           : Set-AZContext $SubID

Position       : At line:7 char:1
                 + Set-AZContext $SubID
                 + ~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
StackTrace     :    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
                    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken)
                    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
                    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
                    at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
                    at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Exception      : System.IO.IOException
InvocationInfo : {Set-AzContext}
Line           : Set-AZContext $SubID

Position       : At line:7 char:1
                 + Set-AZContext $SubID
                 + ~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : An existing connection was forcibly closed by the remote host.
StackTrace     :
Exception      : System.Net.Sockets.SocketException
InvocationInfo : {Set-AzContext}
Line           : Set-AZContext $SubID

Position       : At line:7 char:1
                 + Set-AZContext $SubID
                 + ~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : The SSL connection could not be established, see inner exception.
StackTrace     :    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken
                 cancellationToken)
                    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
                    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
                    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
                    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryAfterDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass15_0.<<SendAsync>b__0>d.MoveNext()
                 --- End of stack trace from previous location ---
                    at Microsoft.Rest.RetryDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Models.RPRegistrationDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.ClaimsChallengeHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Factories.CancelRetryHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts,
                 CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperations.ListWithHttpMessagesAsync(ODataQuery`1 odataQuery, Dictionary`2 customHeaders, CancellationToken cancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperationsExtensions.ListAsync(IResourcesOperations operations, ODataQuery`1 odataQuery, CancellationToken cancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperationsExtensions.List(IResourcesOperations operations, ODataQuery`1 odataQuery)
                    at Microsoft.Azure.Commands.KeyVault.KeyVaultManagementCmdletBase.ListPagable(String resourceGroupName, ODataQuery`1 filter)
                    at Microsoft.Azure.Commands.KeyVault.KeyVaultManagementCmdletBase.ListVaults(String resourceGroupName, Hashtable tag, Nullable`1 resourceTypeName)
                    at Microsoft.Azure.Commands.KeyVault.GetAzureKeyVault.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : System.Net.Http.HttpRequestException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : Authentication failed, see inner exception.
StackTrace     :    at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
                    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken
                 cancellationToken)
Exception      : System.Security.Authentication.AuthenticationException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : The token supplied to the function is invalid
StackTrace     :
Exception      : System.ComponentModel.Win32Exception
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : The SSL connection could not be established, see inner exception.
StackTrace     :    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken
                 cancellationToken)
                    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
                    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
                    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
                    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryAfterDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass15_0.<<SendAsync>b__0>d.MoveNext()
                 --- End of stack trace from previous location ---
                    at Microsoft.Rest.RetryDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Models.RPRegistrationDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.ClaimsChallengeHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Factories.CancelRetryHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts,
                 CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperations.ListWithHttpMessagesAsync(ODataQuery`1 odataQuery, Dictionary`2 customHeaders, CancellationToken cancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperationsExtensions.ListAsync(IResourcesOperations operations, ODataQuery`1 odataQuery, CancellationToken cancellationToken)
                    at Microsoft.Azure.Management.Internal.Resources.ResourcesOperationsExtensions.List(IResourcesOperations operations, ODataQuery`1 odataQuery)
                    at Microsoft.Azure.Commands.KeyVault.KeyVaultManagementCmdletBase.ListPagable(String resourceGroupName, ODataQuery`1 filter)
                    at Microsoft.Azure.Commands.KeyVault.KeyVaultManagementCmdletBase.ListVaults(String resourceGroupName, Hashtable tag, Nullable`1 resourceTypeName)
                    at Microsoft.Azure.Commands.KeyVault.GetAzureKeyVault.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : System.Net.Http.HttpRequestException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
StackTrace     :    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
                    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken)
                    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](CancellationToken cancellationToken)
                    at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
                    at System.Net.Http.ConnectHelper.EstablishSslConnectionAsync(SslClientAuthenticationOptions sslOptions, HttpRequestMessage request, Boolean async, Stream stream, CancellationToken
                 cancellationToken)
Exception      : System.IO.IOException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : An existing connection was forcibly closed by the remote host.
StackTrace     :
Exception      : System.Net.Sockets.SocketException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : An error occurred while sending the request.
StackTrace     :    at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
                    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryAfterDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Rest.RetryDelegatingHandler.<>c__DisplayClass15_0.<<SendAsync>b__0>d.MoveNext()
                 --- End of stack trace from previous location ---
                    at Microsoft.Rest.RetryDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Models.RPRegistrationDelegatingHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.ClaimsChallengeHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at Microsoft.Azure.Commands.Common.Authentication.Factories.CancelRetryHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
                    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts,
                 CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
                    at Microsoft.Azure.Management.KeyVault.VaultsOperations.GetWithHttpMessagesAsync(String resourceGroupName, String vaultName, Dictionary`2 customHeaders, CancellationToken
                 cancellationToken)
                    at Microsoft.Azure.Management.KeyVault.VaultsOperationsExtensions.GetAsync(IVaultsOperations operations, String resourceGroupName, String vaultName, CancellationToken cancellationToken)
                    at Microsoft.Azure.Management.KeyVault.VaultsOperationsExtensions.Get(IVaultsOperations operations, String resourceGroupName, String vaultName)
                    at Microsoft.Azure.Commands.KeyVault.Models.VaultManagementClient.GetVault(String vaultName, String resourceGroupName, IMicrosoftGraphClient graphClient)
                    at Microsoft.Azure.Commands.KeyVault.GetAzureKeyVault.ExecuteCmdlet()
                    at Microsoft.WindowsAzure.Commands.Utilities.Common.AzurePSCmdlet.ProcessRecord()
Exception      : System.Net.Http.HttpRequestException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host..
StackTrace     :    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
                    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Int32>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken)
                    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
                    at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
                    at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
                    at System.Net.Http.HttpConnection.InitialFillAsync(Boolean async)
                    at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
Exception      : System.IO.IOException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
HistoryId      : 40

Message        : An existing connection was forcibly closed by the remote host.
StackTrace     :
Exception      : System.Net.Sockets.SocketException
InvocationInfo : {Get-AzKeyVault}
Line           : $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)

Position       : At line:514 char:21
                 + $KeyVault_Detail = (Get-AzKeyVault $KeyVault.VaultName)
                 +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
chaoscreater commented 1 year ago

OK, I've just set up a fresh install of Windows 11 on a spare laptop. Installed the latest Powershell 7 and AZ module and ran my script while the laptop is connected to router B. I'm getting the same issue as my primary laptop - i.e I still get occasional SSL connection errors. However, I'm also doing a continuous ping test and it hasn't dropped a packet. I've also done a packet loss test (packetlosstest.com) and it's fine, no packet loss at all.

UPDATE 2:

After some more testing, I think I understand this a bit more now. On my laptop running Windows 10 (the one that works without any issues), it's got an Always-On VPN running in the background with no split tunneling. It's actually a work laptop and hence why I'm not fully aware of all the configurations on this machine. All traffic are routed through the VPN tunnel. For some reason, this allows the Powershell script to run without any errors.

On my primary laptop that's having issues, if I connect to any VPN server and run the script, it works fine too.

It makes absolutely no sense. If there was a network related issue with the powerline adapter, or with router B, or with the ethernet cable, then it wouldn't matter if I'm using VPN or not. However, I'm getting consistent successful test results for all laptops when they're all connected to the VPN.

I'm not using any firewall at all either.

UPDATE 3: Today, I was setting up a Linux VM in Azure and I need to install a Powershell module from the PS Gallery (powershellgallery.com). I could use openSSL to connect to powershellgallery.com and also telnet to it on 443, but I kept getting errors about SSL, which is the same error as my original post. There is an Azure Firewall that is used as the default gateway for the Linux VM and after some troubleshooting, I found that some of the Azure edge nodes got blocked by the firewall, so I just had to allow it:

image

And after doing that, everything works.

Going back to my original issue, I'm not using any firewalls, so it's not related. However, the issue goes away when I'm using a VPN. So there must be something that's blocking the traffic. But what's strange is that it only fails when I'm connected to router B and not router A.