search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.57k stars 4.82k forks source link

Managed Identity response was not in the expected format #36216

Closed apetrut closed 1 year ago

apetrut commented 1 year ago

Library name and version

Azure.Identity 1.8.1

Query/Question

Hi,

I created a User Assigned Managed Identity and now I am trying to connect to Azure KeyVault from an ASP.NET Core app deployed as App Service using ManagedIdentityCredential:

var tokenCredentials = new ChainedTokenCredential(
                    new ManagedIdentityCredential("my_client_id")
                                       );
                var client = new SecretClient(new Uri(_kvUri), tokenCredentials);
                var secret = client.GetSecret(secretName, version);

When I run the app I get the error below:

image

However, when I try to connect using DefaultAzureCredential it works:

var tokenCredentials = new ChainedTokenCredential(
                    new DefaultAzureCredential(new DefaultAzureCredentialOptions
                    {
                        ManagedIdentityClientId = "my_client_id"
                    })
                                       );
                var client = new SecretClient(new Uri(_kvUri), tokenCredentials);
                var secret = client.GetSecret(secretName, version);

Can you explain why? I am using the same client id in both scenarios.

Environment

Docker Compose project

jsquire commented 1 year ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

christothes commented 1 year ago

Hi @apetrut - This definitely is strange behavior. Would you mind providing the logging output after reproducing this with logging enabled?

Typically, when you get that exception, it is because some other service is listening on the non-routable ip address that is normally used by the IMDS endpoint (169.254.169.254)

A couple things, unrelated to the specific problem:

github-actions[bot] commented 1 year ago

Hi @apetrut. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

apetrut commented 1 year ago

@christothes The output can be found below. Note that this is happening on my local machine, I don't get the same error when the code is deployed in Azure.

[19:29:36 INF] [KeyVaultService.GetSecret] - Getting the secret from key vault. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.X509Certificates.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.Primitives.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.NonGeneric.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Net.NameResolution.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Net.Mail.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.Specialized.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/RabbitMQ.Client.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Net.Security.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.Identity.Client.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Linq.Expressions.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Azure.Identity.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Azure.Core.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Net.NameResolution.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Threading.Tasks.Extensions.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Azure.Security.KeyVault.Secrets.dll'. Symbols loaded. [19:29:36 INF] [KeyVaultService.GetSecret] - Getting the secret from key vault. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.X509Certificates.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.Primitives.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.NonGeneric.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.Identity.Client.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Linq.Expressions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Net.NameResolution.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.Encoding.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.Encoding.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.Encoding.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Formats.Asn1.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Formats.Asn1.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Formats.Asn1.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Numerics.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Numerics.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Numerics.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.OpenSsl.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.OpenSsl.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Security.Cryptography.OpenSsl.dll'. Cannot find or open the PDB file. [Warning] Azure-Core: Error response [f8f47c16-e738-44b9-9e20-bc0ededf74f8] 401 Unauthorized (01.3s) Cache-Control:no-cache Pragma:no-cache WWW-Authenticate:Bearer authorization="https://login.microsoftonline.com/1f581516-d919-436a-b1cf-dc8b5babd2b5", resource="https://vault.azure.net" x-ms-keyvault-region:uksouth x-ms-client-request-id:f8f47c16-e738-44b9-9e20-bc0ededf74f8 x-ms-request-id:bf018c63-9081-4fa9-843a-a460483c332f x-ms-keyvault-service-version:1.9.804.1 x-ms-keyvault-network-info:conn_type=Ipv4;addr=89.238.217.9;act_addr_fam=InterNetwork; X-Content-Type-Options:REDACTED Strict-Transport-Security:REDACTED Date:Mon, 15 May 2023 19:29:36 GMT Content-Length:97 Content-Type:application/json; charset=utf-8 Expires:-1

[Informational] Azure-Core: Error response [f8f47c16-e738-44b9-9e20-bc0ededf74f8] content: {"error":{"code":"Unauthorized","message":"AKV10000: Request is missing a Bearer or PoP token."}} [Informational] Azure-Identity: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: f8f47c16-e738-44b9-9e20-bc0ededf74f8 Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Extensions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Extensions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Extensions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.TraceSource.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.TraceSource.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.TraceSource.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Serialization.Primitives.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Serialization.Primitives.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Serialization.Primitives.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Xml.ReaderWriter.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Private.Xml.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.IdentityModel.Abstractions.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Microsoft.IdentityModel.Abstractions.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Microsoft.IdentityModel.Abstractions.dll'. Symbols loaded. [Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] MSAL MSAL.NetCore with assembly version '4.46.0.0'. CorrelationId(62d00f88-8a88-4f47-a1ea-cabd957d41a2) [Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] === AcquireTokenForClientParameters === SendX5C: False ForceRefresh: False

[Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] === Request Data === Authority Provided? - True Scopes - https://vault.azure.net/.default Extra Query Params Keys (space separated) - ApiId - AcquireTokenForClient IsConfidentialClient - False SendX5C - False LoginHint ? False IsBrokerConfigured - False HomeAccountId - False CorrelationId - 62d00f88-8a88-4f47-a1ea-cabd957d41a2 UserAssertion set: False LongRunningOboCacheKey set: False Region configured:

[Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] === Token Acquisition (ClientCredentialRequest) started: Scopes: https://vault.azure.net/.default Authority Host: login.microsoftonline.com [Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] Authority validation enabled? True. [Informational] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:37Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] Authority validation - is known env? True. [Informational] Azure-Core: Request [a5985bac-d6f7-45a5-a890-fc2965e39b69] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED&client_id=REDACTED Metadata:REDACTED x-ms-client-request-id:a5985bac-d6f7-45a5-a890-fc2965e39b69 x-ms-return-client-request-id:true User-Agent:azsdk-net-Identity/1.8.1,(.NET 6.0.16; Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022) client assembly: Azure.Identity [Warning] Azure-Core: Error response [a5985bac-d6f7-45a5-a890-fc2965e39b69] 403 connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network. (00.0s) Connection:close

Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Loader.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Loader.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Runtime.Loader.dll'. Cannot find or open the PDB file. Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Private.CoreLib.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Text.Json.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'System.Text.Json.JsonReaderException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.StackTrace.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Reflection.Metadata.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.Immutable.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.StackTrace.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Reflection.Metadata.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.Immutable.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Microsoft.Identity.Client.dll [Error] Azure-Identity: False MSAL 4.46.0.0 MSAL.NetCore .NET 6.0.16 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-05-15 19:29:38Z - 62d00f88-8a88-4f47-a1ea-cabd957d41a2] Exception type: Azure.Identity.CredentialUnavailableException ---> Inner Exception Details Exception type: System.Text.Json.JsonReaderException

at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan1 bytes) at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker) at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first) at System.Text.Json.Utf8JsonReader.ReadSingleSegment() at System.Text.Json.Utf8JsonReader.Read() at System.Text.Json.JsonDocument.Parse(ReadOnlySpan1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack) at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter) at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) === End of inner exception stack trace ===

at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) at Azure.Identity.ImdsManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken) at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentityClient.AuthenticateCoreAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken) Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll at Azure.Identity.ManagedIdentityClient.AppTokenProviderImpl(AppTokenProviderParameters parameters) at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.SendTokenRequestToProviderAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.FetchNewAccessTokenAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.ExecuteAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken) Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Microsoft.Identity.Client.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll [Informational] Azure-Identity: ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: f8f47c16-e738-44b9-9e20-bc0ededf74f8 Exception: Azure.Identity.CredentialUnavailableException (0x80131500): Managed Identity response was not in the expected format. See the inner exception for details. ---> System.Text.Json.JsonReaderException (0x80131500): 'c' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Diagnostics.StackTrace.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Reflection.Metadata.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.Collections.Immutable.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Microsoft.Identity.Client.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Security.KeyVault.Secrets.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.IO.MemoryMappedFiles.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll [19:29:38 ERR] [KeyVaultService.GetSecret] - Could not fetch secret. Azure.Identity.CredentialUnavailableException: Managed Identity response was not in the expected format. See the inner exception for details. ---> System.Text.Json.JsonReaderException: 'c' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0. at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan1 bytes) at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker) at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first) at System.Text.Json.Utf8JsonReader.ReadSingleSegment() at System.Text.Json.Utf8JsonReader.Read() at System.Text.Json.JsonDocument.Parse(ReadOnlySpan1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack) at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter) at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Azure.Identity.ChainedTokenCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Identity.ChainedTokenCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallenge(HttpMessage message) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken) Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TResult](RequestMethod method, Func1 resultFactory, CancellationToken cancellationToken, String[] path) at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken) at ADC.MultiTenancy.Services.KeyVaultService.GetSecret(String secretName, String version) in C:\Users\AndreiPetrut\source\repos\htc.identityserver\HTC.IdentityServer\Src\ADC.MultiTenancy\Services\KeyVaultService.cs:line 54 Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.EntityFramework.Configuration.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Skoruba.AuditLogging.EntityFramework.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Security.KeyVault.Secrets.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.IO.MemoryMappedFiles.dll'. Cannot find or open the PDB file. [19:29:38 ERR] [KeyVaultService.GetSecret] - Could not fetch secret. Azure.Identity.CredentialUnavailableException: Managed Identity response was not in the expected format. See the inner exception for details. ---> System.Text.Json.JsonReaderException: 'c' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0. at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan1 bytes) at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker) at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first) at System.Text.Json.Utf8JsonReader.ReadSingleSegment() at System.Text.Json.Utf8JsonReader.Read() at System.Text.Json.JsonDocument.Parse(ReadOnlySpan1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack) at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter) Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Azure.Identity.ChainedTokenCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Identity.ChainedTokenCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallenge(HttpMessage message) Loaded '/app/bin/Debug/net6.0/Azure.Extensions.AspNetCore.DataProtection.Keys.dll'. Symbols loaded. at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TResult](RequestMethod method, Func1 resultFactory, CancellationToken cancellationToken, String[] path) at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken) at ADC.MultiTenancy.Services.KeyVaultService.GetSecret(String secretName, String version) in C:\Users\AndreiPetrut\source\repos\htc.identityserver\HTC.IdentityServer\Src\ADC.MultiTenancy\Services\KeyVaultService.cs:line 54 Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Identity.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Diagnostics.HealthChecks.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.EntityFramework.Configuration.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.HttpsPolicy.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.BusinessLogic.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Identity.Core.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Abstractions.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.Abstractions.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cryptography.Internal.dll'. Cannot find or open the PDB file. Exception thrown: 'Azure.Identity.CredentialUnavailableException' in System.Private.CoreLib.dll Loaded '/app/bin/Debug/net6.0/Duende.IdentityServer.Storage.dll'. Exception thrown: 'System.DllNotFoundException' in Microsoft.AspNetCore.Cryptography.Internal.dll Exception thrown: 'Azure.Identity.CredentialUnavailableException' in Azure.Security.KeyVault.Secrets.dll Loaded '/app/bin/Debug/net6.0/SendGrid.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Identity.Core.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/6.0.16/System.IO.MemoryMappedFiles.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.dll'. Cannot find or open the PDB file. [19:29:38 ERR] [KeyVaultService.GetSecret] - Could not fetch secret. Azure.Identity.CredentialUnavailableException: Managed Identity response was not in the expected format. See the inner exception for details. ---> System.Text.Json.JsonReaderException: 'c' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0. at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan1 bytes) at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker) at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first) at System.Text.Json.Utf8JsonReader.ReadSingleSegment() at System.Text.Json.Utf8JsonReader.Read() at System.Text.Json.JsonDocument.Parse(ReadOnlySpan1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack) at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter) at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken) at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken) --- End of inner exception stack trace --- at Azure.Identity.ChainedTokenCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Identity.ChainedTokenCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async) at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async) Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.CookiePolicy.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Http.Features.dll'. Cannot find or open the PDB file. at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallenge(HttpMessage message) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RedirectPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async) at Azure.Core.Pipeline.RetryPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelinePolicy.ProcessNext(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.Process(HttpMessage message, ReadOnlyMemory1 pipeline) at Azure.Core.Pipeline.HttpPipeline.Send(HttpMessage message, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest(Request request, CancellationToken cancellationToken) at Azure.Security.KeyVault.KeyVaultPipeline.SendRequest[TResult](RequestMethod method, Func1 resultFactory, CancellationToken cancellationToken, String[] path) at Azure.Security.KeyVault.Secrets.SecretClient.GetSecret(String name, String version, CancellationToken cancellationToken) at ADC.MultiTenancy.Services.KeyVaultService.GetSecret(String secretName, String version) in C:\Users\AndreiPetrut\source\repos\htc.identityserver\HTC.IdentityServer\Src\ADC.MultiTenancy\Services\KeyVaultService.cs:line 54 Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.EntityFramework.Configuration.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Cookies.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Core.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/HTC.IdentityServer.Admin.EntityFramework.SqlServer.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.WebEncoders.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Azure.Extensions.AspNetCore.DataProtection.Keys.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Azure.Extensions.AspNetCore.DataProtection.Keys.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.DataProtection.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cryptography.Internal.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.Identity.Web.dll'. Symbols loaded. Exception thrown: 'System.DllNotFoundException' in Microsoft.AspNetCore.Cryptography.Internal.dll Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Authentication.OpenIdConnect.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/AspNet.Security.OAuth.GitHub.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cryptography.Internal.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.OAuth.dll'. Cannot find or open the PDB file. Exception thrown: 'System.DllNotFoundException' in Microsoft.AspNetCore.Cryptography.Internal.dll Loaded '/app/bin/Debug/net6.0/Duende.IdentityServer.EntityFramework.dll'. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.CookiePolicy.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Identity.UI.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Duende.IdentityServer.AspNetIdentity.dll'. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Http.Features.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Identity.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/SendGrid.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Duende.IdentityServer.Storage.dll'. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Cookies.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Identity.Core.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Identity.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Authentication.OpenIdConnect.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Core.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.WebEncoders.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Http.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Authentication.JwtBearer.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cors.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authentication.Core.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.WebEncoders.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.EntityFramework.Identity.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Memory.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.Policy.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Abstractions.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/AutoMapper.dll'. Symbols loaded. Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.EntityFramework.Identity.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.ViewFeatures.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.HttpsPolicy.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cors.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Razor.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.DataAnnotations.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.DataAnnotations.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Localization.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Razor.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.ApiExplorer.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.DataAnnotations.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Mvc.Razor.RuntimeCompilation.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.Policy.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Localization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Cors.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.JSInterop.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.Web.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Antiforgery.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.TagHelpers.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.AspNetCore.Identity.UI.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Razor.Runtime.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/AutoMapper.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.ApiExplorer.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Authorization.Policy.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/ADC.Cryptography.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Cors.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Cors.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Skoruba.Duende.IdentityServer.Admin.UI.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Diagnostics.HealthChecks.Abstractions.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.JSInterop.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Razor.Runtime.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.Web.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Razor.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Antiforgery.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.ApiExplorer.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/HealthChecks.SqlServer.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Razor.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.Cors.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Abstractions.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.ViewFeatures.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Memory.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.TagHelpers.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.JSInterop.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Components.Web.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Antiforgery.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Razor.dll'. Cannot find or open the PDB file. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Abstractions.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.Extensions.Caching.Memory.dll'. Symbols loaded. Loaded '/usr/share/dotnet/shared/Microsoft.AspNetCore.App/6.0.16/Microsoft.AspNetCore.Mvc.TagHelpers.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/Swashbuckle.AspNetCore.Swagger.dll'. Cannot find or open the PDB file. Loaded '/app/bin/Debug/net6.0/runtimes/unix/lib/netcoreapp3.1/Microsoft.Data.SqlClient.dll'. Cannot find or open the PDB file.

christothes commented 1 year ago

Note that this is happening on my local machine, I don't get the same error when the code is deployed in Azure.

Ah - this changes everything 😄 This means that your local network has something listening on the IMDS endpoint. This will always result in some kind of exception, but CredentialUnavailableException is most common and is expected. The down side is that it causes potentially long delays for that initial credential discovery.

If the process listening on the IMDS endpoint can't be changed, one possible workaround is to set the AZURE_POD_IDENTITY_AUTHORITY_HOST environment variable to an IP address that you are sure has no listener on your dev machine.

github-actions[bot] commented 1 year ago

Hi @apetrut. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

apetrut commented 1 year ago

This means that your local network has something listening on the IMDS endpoint

I am not aware of any app listening on that endpoint, how can I check that?

Where should I set this ENV variable (AZURE_POD_IDENTITY_AUTHORITY_HOST)?

Is there any documentation about it?

christothes commented 1 year ago

This means that your local network has something listening on the IMDS endpoint

I am not aware of any app listening on that endpoint, how can I check that?

Yes, you can see processes listening on port 80 a few different ways. Here is one

Where should I set this ENV variable (AZURE_POD_IDENTITY_AUTHORITY_HOST)?

Locally, you'd just set it in your windows system properties image

Is there any documentation about it?

Not really, but this allows you to customize the hostname used to connect to the IMDS managed identity endpoint.

github-actions[bot] commented 1 year ago

Hi @apetrut. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

apetrut commented 1 year ago

I checked and I've got Docker and wslhost processes listening on port 80. I need those processes because my app is running in Docker.

However none of these processes seem to be listening on the IMDS endpoint:

image

christothes commented 1 year ago

Sorry, I didn't realize this was in docker. Please see this issue for some details on this scenario. Right now there are a few options to get managed identity to work locally, but none of them are great.

github-actions[bot] commented 1 year ago

Hi @apetrut. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

github-actions[bot] commented 1 year ago

Hi @apetrut, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!