search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.47k stars 4.8k forks source link

[BUG] Azure Identity 1.9.0 but fails in v 1.10 fails #38218

Closed BC89 closed 1 year ago

BC89 commented 1 year ago

Library name and version

Azure.Identity v1.10.0

Describe the bug

This code works in Azure Identity 1.9.0 but fails in v 1.10 fails:

var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions
{
    TenantId = tenantid,
    ManagedIdentityClientId = managedidentityclientid,
});

var token = credential.GetToken(new TokenRequestContext(new string[] { $"https://{keyvaultname}.vault.azure.net/" }));
var client = new SecretClient(new Uri($"https://{keyvaultname}.vault.azure.net/"), credential);

If I upgrade the nuget and rollback w/out any other changes in the codebase this breaks every time. The error is as follows:

Azure.Identity.AuthenticationFailedException: 'ManagedIdentityCredential authentication failed: Service request failed.
Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.)

Expected behavior

Functionality would not break.

Actual behavior

Does not work.

Reproduction Steps

Upgrade the nuget and watch it fail.

Environment

<PackageReference Include="Azure.Identity" Version="1.10.0" />
<PackageReference Include="Azure.Monitor.OpenTelemetry.AspNetCore" Version="1.0.0-beta.6" />
<PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="7.0.10" />
<PackageReference Include="Microsoft.VisualStudio.Azure.Containers.Tools.Targets" Version="1.19.5" />
<PackageReference Include="Swashbuckle.AspNetCore" Version="6.5.0" />
<PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.5.0" />
net7.0

Running inside a docker-compose on windows but linux containers. Have logged in between sessions to ensure local token cache is up to date from within VS 2022 IDE Account Settings -

Microsoft Visual Studio Professional 2022 (64-bit) - Preview Version 17.8.0 Preview 1.0

christothes commented 1 year ago

Hi @BC89 - Would you mind providing the logging output after reproducing this with logging enabled?

github-actions[bot] commented 1 year ago

Hi @BC89. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

BC89 commented 1 year ago 
An unhandled exception of type 'Azure.Identity.AuthenticationFailedException' occurred in System.Private.CoreLib.dll: 'ManagedIdentityCredential authentication failed: Service request failed.
Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.)

Headers:
Connection: close

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot'
Stack trace:
 >   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
 >   at Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__16.MoveNext()
 >   at System.Threading.Tasks.ValueTask`1.get_Result()
 >   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
 >   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
 >   at Azure.Identity.ManagedIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
 >   at Azure.Identity.DefaultAzureCredential.<GetTokenFromSourcesAsync>d__14.MoveNext()
 >   at System.Threading.Tasks.ValueTask`1.get_Result()
 >   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
 >   at Azure.Identity.DefaultAzureCredential.<GetTokenImplAsync>d__12.MoveNext()
 >   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
 >   at Azure.Identity.DefaultAzureCredential.<GetTokenImplAsync>d__12.MoveNext()
 >   at System.Threading.Tasks.ValueTask`1.get_Result()
 >   at System.Runtime.CompilerServices.ValueTaskAwaiter`1.GetResult()
 >   at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
 >   at Azure.Identity.DefaultAzureCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
 >   at Program.<Main>$(String[] args) in C:\Users\brendan.carroll\source\repos\Bryx\Marketplace-Services\Services\Email\Program.cs:line 37
Loaded '/app/bin/Debug/net7.0/Microsoft.Identity.Client.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
BC89 commented 1 year ago 
[Informational] Azure-Identity: DefaultAzureCredential.GetToken invoked. Scopes: [ https://k......vault.azure.net/ ] ParentRequestId: 
[Informational] Azure-Identity: EnvironmentCredential.GetToken invoked. Scopes: [ https://k.......vault.azure.net/ ] ParentRequestId: 
[Informational] Azure-Identity: EnvironmentCredential.GetToken was unable to retrieve an access token. Scopes: [ https://k.......vault.azure.net/ ] ParentRequestId:  Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
[Informational] Azure-Identity: WorkloadIdentityCredential.GetToken invoked. Scopes: [ https://k.......vault.azure.net/ ] ParentRequestId: 
[Informational] Azure-Identity: WorkloadIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://k.......vault.azure.net/ ] ParentRequestId:  Exception: Azure.Identity.CredentialUnavailableException (0x80131500): WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot
[Informational] Azure-Identity: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://k.......vault.azure.net/ ] ParentRequestId: 
[Informational] Azure-Identity: False MSAL 4.54.1.0 MSAL.NetCore .NET 7.0.10 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-08-15 20:37:14Z - e4997ec2-79f6-4a66-9b91-69761dfeeadd] MSAL MSAL.NetCore with assembly version '4.54.1.0'. CorrelationId(e4997ec2-79f6-4a66-9b91-69761dfeeadd)
[Informational] Azure-Identity: False MSAL 4.54.1.0 MSAL.NetCore .NET 7.0.10 Linux 5.10.102.1-microsoft-standard-WSL2 #1 SMP Wed Mar 2 00:30:59 UTC 2022 [2023-08-15 20:37:14Z - e4997ec2-79f6-4a66-9b91-69761dfeeadd] === AcquireTokenForClientParameters ===
SendX5C: False
ForceRefresh: False
thremendus commented 1 year ago

I am also experiencing the same issue. I just upgraded from 1.9.0 to 1.10.0 and started getting a 403 exception when using DefaultCredentials() against AzureKeyVault. I triple checked my permissions and my user is fine. I downgraded back to 1.9.0 and it works again. Worth mentioning I run my .NET 7 Web API from VS 2022 Enterprise in a docker container (linux).

Code in question:

builder.Configuration.AddAzureKeyVault(
       new Uri($"{builder.Configuration["AzureKeyVaultConfiguration:BaseUrl"]}"),
       new DefaultAzureCredential(),
       new AzureKeyVaultConfigurationOptions
       {       
           ReloadInterval = TimeSpan.FromHours(24)
       });

Error:

Service request failed.
Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.)

Headers:
Connection: close

   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
   at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
   at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(TokenRequestContext context, Boolean async, CancellationToken cancellationToken)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, TokenRequestContext context, Boolean async)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AuthenticateAndAuthorizeRequestAsync(HttpMessage message, TokenRequestContext context)
   at Azure.Security.KeyVault.ChallengeBasedAuthenticationPolicy.AuthorizeRequestOnChallengeAsyncInternal(HttpMessage message, Boolean async)
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
   at Azure.Core.Pipeline.HttpPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.SendRequestAsync(Request request, CancellationToken cancellationToken)
   at Azure.Security.KeyVault.KeyVaultPipeline.GetPageAsync[T](Uri firstPageUri, String nextLink, Func`1 itemFactory, String operationName, CancellationToken cancellationToken)
   at Azure.Core.PageResponseEnumerator.FuncAsyncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+MoveNext()
   at Azure.Core.PageResponseEnumerator.FuncAsyncPageable`1.AsPages(String continuationToken, Nullable`1 pageSizeHint)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at Azure.AsyncPageable`1.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Azure.Extensions.AspNetCore.Configuration.Secrets.AzureKeyVaultConfigurationProvider.LoadAsync()
   at Azure.Extensions.AspNetCore.Configuration.Secrets.AzureKeyVaultConfigurationProvider.LoadAsync()
   at Azure.Extensions.AspNetCore.Configuration.Secrets.AzureKeyVaultConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationManager.AddSource(IConfigurationSource source)
   at Microsoft.Extensions.Configuration.ConfigurationManager.Microsoft.Extensions.Configuration.IConfigurationBuilder.Add(IConfigurationSource source)
   at Microsoft.Extensions.Configuration.AzureKeyVaultConfigurationExtensions.AddAzureKeyVault(IConfigurationBuilder configurationBuilder, SecretClient client, AzureKeyVaultConfigurationOptions options)
   at Microsoft.Extensions.Configuration.AzureKeyVaultConfigurationExtensions.AddAzureKeyVault(IConfigurationBuilder configurationBuilder, Uri vaultUri, TokenCredential credential, AzureKeyVaultConfigurationOptions options)
   at Program.<Main>$(String[] args) in C:\Develop\Elephant-Api\Elephant.WebApi\Program.cs:line 36

Screenshot of the code breaking:

image

christothes commented 1 year ago

Thanks for the additional information.

@BC89 - I have a few things that would help isolate the issue, if you'd be willing to try them. I'll also work on setting up a repro locally, but these things may help short circuit the investigation.

@thremendus - If you are willing to try the same, please do.

github-actions[bot] commented 1 year ago

Hi @BC89. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

christothes commented 1 year ago

Just a quick update - I have a local repro and understand the reason we are seeing this new behavior. I'll update this issue when we decide on how to address this behavior.

christothes commented 1 year ago

We're still deciding how we want to address this issue, since the issue seems to be how Docker Desktop response to requests to an unreachable network. For example, here is a curl command to the IMDS endpoint run on my repro machine across 3 different environments

curl -I http://169.254.169.254

On Windows: curl: (7) Failed to connect to 169.254.169.254 port 80 after 0 ms: Couldn't connect to server

On WSL2 Ubuntu: curl: (28) Failed to connect to 169.254.169.254 port 80: Connection timed out

On Docker Desktop from a Linux container: HTTP/1.0 403 connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network. Connection: close

But, as a workaround, when running in the local environment, excluding the ManagedIdentityCredential should avoid this problem.

var options = new DefaultAzureCredentialOptions { ExcludeManagedIdentityCredential = true };
var credential = new DefaultAzureCredential(options);
KijoKim76 commented 1 year ago

Thanks. I had the same issue for Open AI + Cognitive Search but christothes's fix seems to working. https://github.com/Azure-Samples/azure-search-openai-demo-csharp/tree/feature/embeddingSearch Thanks

RobertPaulson90 commented 1 year ago

Why was this closed? It is still a major issue for me and my team.

RobertPaulson90 commented 1 year ago

But, as a workaround, when running in the local environment, excluding the ManagedIdentityCredential should avoid this problem.

This workaround is not viable in Python when my Function input trigger is service bus using managed identity. I guess I need to start using connection strings in the meantime, until a fix is provided?

christothes commented 1 year ago

Why was this closed? It is still a major issue for me and my team.

This should be fixed in version 1.10.1 , which was recently released.

aconsolati commented 11 months ago

This is not working when attempting to connect to SQL Azure from within a docker-compose project using Authentication=Active Directory Default in the connection string. The same code works fine when targetting the local Windows OS.

Dockerfile base image: mcr.microsoft.com/dotnet/sdk:8.0 Microsoft Visual Studio Enterprise 2022 (64-bit) - Current Version 17.8.1

<TargetFramework>net8.0</TargetFramework> <PackageReference Include="Azure.Identity" Version="1.10.4" />

Connection String: Server=tcp:{redacted}.database.windows.net,1433;Authentication=Active Directory Default;Database={redacted};

docker-compose output stack-trace (abbreviated):

1>#18 8.395 See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
1>#18 8.395  ---> Azure.RequestFailedException: Managed Identity response was not in the expected format. See the inner exception for details.
1>#18 8.395 Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable network.)
1>#18 8.395
1>#18 8.395 Headers:
1>#18 8.395 Connection: close
1>#18 8.395
1>#18 8.395  ---> System.Text.Json.JsonReaderException: 'c' is an invalid start of a value. LineNumber: 0 | BytePositionInLine: 0.
1>#18 8.395    at System.Text.Json.ThrowHelper.ThrowJsonReaderException(Utf8JsonReader& json, ExceptionResource resource, Byte nextByte, ReadOnlySpan`1 bytes)
1>#18 8.395    at System.Text.Json.Utf8JsonReader.ConsumeValue(Byte marker)
1>#18 8.395    at System.Text.Json.Utf8JsonReader.ReadFirstToken(Byte first)
1>#18 8.395    at System.Text.Json.Utf8JsonReader.ReadSingleSegment()
1>#18 8.395    at System.Text.Json.Utf8JsonReader.Read()
1>#18 8.395    at System.Text.Json.JsonDocument.Parse(ReadOnlySpan`1 utf8JsonSpan, JsonReaderOptions readerOptions, MetadataDb& database, StackRowStack& stack)
1>#18 8.395    at System.Text.Json.JsonDocument.Parse(ReadOnlyMemory`1 utf8Json, JsonReaderOptions readerOptions, Byte[] extraRentedArrayPoolBytes, PooledByteBufferWriter extraPooledByteBufferWriter)
1>#18 8.395    at System.Text.Json.JsonDocument.ParseAsyncCore(Stream utf8Json, JsonDocumentOptions options, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken)
1>#18 8.395    --- End of inner exception stack trace ---
1>#18 8.395    at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ImdsManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, Response response, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    --- End of inner exception stack trace ---
1>#18 8.395    at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
1>#18 8.395    at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.ManagedIdentityCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage)
1>#18 8.395    at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
1>#18 8.395    at Microsoft.Data.SqlClient.ActiveDirectoryAuthenticationProvider.AcquireTokenAsync(SqlAuthenticationParameters parameters)
1>#18 8.395    at Microsoft.Data.SqlClient.SqlInternalConnectionTds.<>c__DisplayClass147_1.<<GetFedAuthToken>b__1>d.MoveNext()
...
BC89 commented 10 months ago

Seems like there was a regression here. Running Microsoft Visual Studio Professional 2022 (64-bit) - Preview Version 17.9.0 Preview 2.0 with Azure.Identity 1.10.4 and Azure.Security.KeyVault.Secrets 4.5.0

As of a few days ago started getting the 403 error while attempting to get a token from KeyVault. Fixed by simply adding ExcludeManagedIdentityCredential = true but prior it was working fine without. Removing/adding ExcludeManagedIdentityCredential is like an feature error switch now :)

Running inside a docker-compose on windows but linux containers. .Net 8 -FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base

christothes commented 10 months ago

Seems like there was a regression here. Running Microsoft Visual Studio Professional 2022 (64-bit) - Preview Version 17.9.0 Preview 2.0 with Azure.Identity 1.10.4 and Azure.Security.KeyVault.Secrets 4.5.0

As of a few days ago started getting the 403 error while attempting to get a token from KeyVault. Fixed by simply adding ExcludeManagedIdentityCredential = true but prior it was working fine without. Removing/adding ExcludeManagedIdentityCredential is like an feature error switch now :)

Running inside a docker-compose on windows but linux containers. .Net 8 -FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base

@BC89 - Could you share the details of the exception including the full error message?

ockert-pretorius commented 10 months ago

Im experiencing a similar issue when connecting to EventHubs using the Function Binding EventHubTrigger with an Identity based connection string. The error appears to be related to ManagedIdentity. When running the function locally, the error is logged as a Credential failure, but the credential is successfully obtained via VisualStudioCredential and the function continues to work as expected. However, once the function is deployed, it stops working because the server uses ManagedIdentityCredential. Reverting Azure.Identity back to version 1.9.0 resolves the issue


[2024-01-11T11:05:23.175Z] EnvironmentCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:23.180Z] EnvironmentCredential.GetToken was unable to retrieve an access token. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:  Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
[2024-01-11T11:05:23.188Z] WorkloadIdentityCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:23.190Z] WorkloadIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:  Exception: Azure.Identity.CredentialUnavailableException (0x80131500): WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/workloadidentitycredential/troubleshoot
[2024-01-11T11:05:23.196Z] ManagedIdentityCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:23.259Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] MSAL MSAL.NetCore with assembly version '4.56.0.0'. CorrelationId(c4cb8c80-5109-4966-8596-5859a79390a8)
[2024-01-11T11:05:23.275Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] === AcquireTokenForClientParameters ===
[2024-01-11T11:05:23.278Z] SendX5C: False
[2024-01-11T11:05:23.279Z] ForceRefresh: False
[2024-01-11T11:05:23.280Z]
[2024-01-11T11:05:23.287Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8]
[2024-01-11T11:05:23.288Z] === Request Data ===
[2024-01-11T11:05:23.289Z] Authority Provided? - True
[2024-01-11T11:05:23.290Z] Scopes - https://eventhubs.azure.net/.default
[2024-01-11T11:05:23.291Z] Extra Query Params Keys (space separated) -
[2024-01-11T11:05:23.294Z] ApiId - AcquireTokenForClient
[2024-01-11T11:05:23.296Z] IsConfidentialClient - True
[2024-01-11T11:05:23.296Z] SendX5C - False
[2024-01-11T11:05:23.297Z] LoginHint ? False
[2024-01-11T11:05:23.298Z] IsBrokerConfigured - False
[2024-01-11T11:05:23.299Z] HomeAccountId - False
[2024-01-11T11:05:23.300Z] CorrelationId - c4cb8c80-5109-4966-8596-5859a79390a8
[2024-01-11T11:05:23.301Z] UserAssertion set: False
[2024-01-11T11:05:23.301Z] LongRunningOboCacheKey set: False
[2024-01-11T11:05:23.302Z] Region configured:
[2024-01-11T11:05:23.303Z]
[2024-01-11T11:05:23.305Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] === Token Acquisition (ClientCredentialRequest) started:
         Scopes: https://eventhubs.azure.net/.default
        Authority Host: login.microsoftonline.com
[2024-01-11T11:05:23.331Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] [Region discovery] Not using a regional authority.
[2024-01-11T11:05:23.333Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] [Instance Discovery] Skipping Instance discovery because it is disabled.
[2024-01-11T11:05:23.337Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:23Z - c4cb8c80-5109-4966-8596-5859a79390a8] [ClientCredentialRequest] Sending Token response to client credential request endpoint ...
[2024-01-11T11:05:23.352Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
[2024-01-11T11:05:23.358Z] Metadata:REDACTED
[2024-01-11T11:05:23.359Z] x-ms-client-request-id:40f919aa-9f8c-4bb7-bae1-7be24d269181
[2024-01-11T11:05:23.361Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:23.362Z] User-Agent:azsdk-net-Identity/1.10.4 (.NET 6.0.24; Microsoft Windows 10.0.22621)[2024-01-11T11:05:23.363Z] client assembly: Azure.Identity
[2024-01-11T11:05:23.458Z] Retrieving properties for Event Hub: xxxxxxxxxxxx.
[2024-01-11T11:05:23.462Z] DefaultAzureCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:23.493Z] Request [55e1ed99-2a07-420f-abcd-9354df9c6c4d] HEAD http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-eventhub/xxxxxxxxxxxx.servicebus.windows.net/xxxxxxxxxxxx/eme/checkpoint/-1
[2024-01-11T11:05:23.496Z] x-ms-version:2022-11-02
[2024-01-11T11:05:23.496Z] Accept:application/xml
[2024-01-11T11:05:23.497Z] x-ms-client-request-id:55e1ed99-2a07-420f-abcd-9354df9c6c4d
[2024-01-11T11:05:23.498Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:23.499Z] User-Agent:azsdk-net-Storage.Blobs/12.16.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:23.500Z] x-ms-date:Thu, 11 Jan 2024 11:05:23 GMT
[2024-01-11T11:05:23.501Z] Authorization:REDACTED
[2024-01-11T11:05:23.502Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:23.554Z] Error response [55e1ed99-2a07-420f-abcd-9354df9c6c4d] 404 The specified blob does not exist. (00.1s)
[2024-01-11T11:05:23.556Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:23.557Z] x-ms-error-code:BlobNotFound
[2024-01-11T11:05:23.558Z] x-ms-request-id:52fd0618-632b-4aca-b302-3b73a8489fc7
[2024-01-11T11:05:23.559Z] Date:Thu, 11 Jan 2024 11:05:23 GMT
[2024-01-11T11:05:23.560Z] Connection:keep-alive
[2024-01-11T11:05:23.561Z] Keep-Alive:REDACTED
[2024-01-11T11:05:23.562Z]
[2024-01-11T11:05:23.596Z] Request [4ca7c86f-1a84-47b8-956b-c9f4e14bdcaf] GET http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-eventhub/xxxxxxxxxxxx.servicebus.windows.net/xxxxxxxxxxxx/eme/-1
[2024-01-11T11:05:23.602Z] x-ms-range:bytes=0-268435455
[2024-01-11T11:05:23.602Z] x-ms-version:2022-11-02
[2024-01-11T11:05:23.603Z] Accept:application/xml
[2024-01-11T11:05:23.604Z] x-ms-client-request-id:4ca7c86f-1a84-47b8-956b-c9f4e14bdcaf
[2024-01-11T11:05:23.605Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:23.606Z] User-Agent:azsdk-net-Storage.Blobs/12.16.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:23.607Z] x-ms-date:Thu, 11 Jan 2024 11:05:23 GMT
[2024-01-11T11:05:23.608Z] Authorization:REDACTED
[2024-01-11T11:05:23.609Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:23.664Z] Error response [4ca7c86f-1a84-47b8-956b-c9f4e14bdcaf] 404 The specified blob does not exist. (00.1s)
[2024-01-11T11:05:23.666Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:23.667Z] x-ms-error-code:BlobNotFound
[2024-01-11T11:05:23.668Z] x-ms-request-id:b7279030-f3fc-4fa8-be21-84addba08fc1
[2024-01-11T11:05:23.669Z] Date:Thu, 11 Jan 2024 11:05:23 GMT
[2024-01-11T11:05:23.670Z] Connection:keep-alive
[2024-01-11T11:05:23.670Z] Keep-Alive:REDACTED
[2024-01-11T11:05:23.672Z] Transfer-Encoding:chunked
[2024-01-11T11:05:23.672Z] Content-Type:application/xml
[2024-01-11T11:05:23.674Z]
[2024-01-11T11:05:24.427Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] exception System.Threading.Tasks.TaskCanceledException: The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.
[2024-01-11T11:05:24.430Z]  ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
[2024-01-11T11:05:24.431Z]    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
[2024-01-11T11:05:24.433Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-01-11T11:05:24.434Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-01-11T11:05:24.435Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-01-11T11:05:24.436Z]    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:24.437Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:24.439Z]    --- End of inner exception stack trace ---
[2024-01-11T11:05:24.440Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)
[2024-01-11T11:05:24.441Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:24.443Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:25.268Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] attempt number 1 took 01.1s
[2024-01-11T11:05:25.275Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
[2024-01-11T11:05:25.277Z] Metadata:REDACTED
[2024-01-11T11:05:25.278Z] x-ms-client-request-id:40f919aa-9f8c-4bb7-bae1-7be24d269181
[2024-01-11T11:05:25.279Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:25.282Z] User-Agent:azsdk-net-Identity/1.10.4 (.NET 6.0.24; Microsoft Windows 10.0.22621)[2024-01-11T11:05:25.283Z] client assembly: Azure.Identity
[2024-01-11T11:05:26.315Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] exception System.Threading.Tasks.TaskCanceledException: The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.
[2024-01-11T11:05:26.316Z]  ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
[2024-01-11T11:05:26.318Z]    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
[2024-01-11T11:05:26.319Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-01-11T11:05:26.320Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-01-11T11:05:26.321Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-01-11T11:05:26.322Z]    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:26.323Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:26.325Z]    --- End of inner exception stack trace ---
[2024-01-11T11:05:26.325Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)
[2024-01-11T11:05:26.326Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:26.327Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:27.368Z] Request [6ad14344-40dc-41bb-9ee5-5ec95a787f4f] HEAD http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-hosts/locks/csjhbn89vzjg3-2090646957/host
[2024-01-11T11:05:27.376Z] x-ms-version:2021-08-06
[2024-01-11T11:05:27.382Z] Accept:application/xml
[2024-01-11T11:05:27.386Z] x-ms-client-request-id:6ad14344-40dc-41bb-9ee5-5ec95a787f4f
[2024-01-11T11:05:27.386Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:27.387Z] User-Agent:azsdk-net-Storage.Blobs/12.13.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:27.388Z] x-ms-date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.389Z] Authorization:REDACTED
[2024-01-11T11:05:27.390Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:27.432Z] Response [6ad14344-40dc-41bb-9ee5-5ec95a787f4f] 200 OK (00.0s)
[2024-01-11T11:05:27.433Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:27.434Z] x-ms-creation-time:Fri, 22 Dec 2023 10:17:04 GMT
[2024-01-11T11:05:27.436Z] x-ms-meta-FunctionInstance:REDACTED
[2024-01-11T11:05:27.437Z] x-ms-blob-type:BlockBlob
[2024-01-11T11:05:27.437Z] x-ms-lease-state:expired
[2024-01-11T11:05:27.438Z] x-ms-lease-status:unlocked
[2024-01-11T11:05:27.439Z] ETag:"0x253ADA9DEF85160"
[2024-01-11T11:05:27.440Z] x-ms-client-request-id:6ad14344-40dc-41bb-9ee5-5ec95a787f4f
[2024-01-11T11:05:27.441Z] x-ms-request-id:da60a674-bcf8-4b62-9a2e-d2fcf93e0eae
[2024-01-11T11:05:27.442Z] x-ms-version:2023-08-03
[2024-01-11T11:05:27.443Z] Date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.444Z] Accept-Ranges:bytes
[2024-01-11T11:05:27.445Z] x-ms-server-encrypted:true
[2024-01-11T11:05:27.445Z] x-ms-access-tier:Hot
[2024-01-11T11:05:27.446Z] x-ms-access-tier-inferred:true
[2024-01-11T11:05:27.447Z] x-ms-access-tier-change-time:Fri, 22 Dec 2023 10:17:04 GMT
[2024-01-11T11:05:27.448Z] Connection:keep-alive
[2024-01-11T11:05:27.449Z] Keep-Alive:REDACTED
[2024-01-11T11:05:27.449Z] Last-Modified:Thu, 11 Jan 2024 11:03:51 GMT
[2024-01-11T11:05:27.450Z] Content-Length:0
[2024-01-11T11:05:27.451Z] Content-Type:application/octet-stream
[2024-01-11T11:05:27.452Z] Content-MD5:1B2M2Y8AsgTpgAmY7PhCfg==
[2024-01-11T11:05:27.453Z]
[2024-01-11T11:05:27.465Z] Request [1102e74e-0c26-474c-8749-50e136be1a31] PUT http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-hosts/locks/csjhbn89vzjg3-2090646957/host?comp=lease
[2024-01-11T11:05:27.466Z] x-ms-lease-action:acquire
[2024-01-11T11:05:27.467Z] x-ms-lease-duration:15
[2024-01-11T11:05:27.469Z] x-ms-proposed-lease-id:000000000000000000000000CD5B5480
[2024-01-11T11:05:27.470Z] x-ms-version:2021-08-06
[2024-01-11T11:05:27.471Z] Accept:application/xml
[2024-01-11T11:05:27.472Z] x-ms-client-request-id:1102e74e-0c26-474c-8749-50e136be1a31
[2024-01-11T11:05:27.473Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:27.475Z] User-Agent:azsdk-net-Storage.Blobs/12.13.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:27.476Z] x-ms-date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.477Z] Authorization:REDACTED
[2024-01-11T11:05:27.477Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:27.522Z] Response [1102e74e-0c26-474c-8749-50e136be1a31] 201 Created (00.0s)
[2024-01-11T11:05:27.524Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:27.525Z] ETag:"0x253ADA9DEF85160"
[2024-01-11T11:05:27.526Z] x-ms-lease-id:000000000000000000000000CD5B5480
[2024-01-11T11:05:27.527Z] x-ms-client-request-id:1102e74e-0c26-474c-8749-50e136be1a31
[2024-01-11T11:05:27.528Z] x-ms-request-id:3c62ce2a-36fd-4b03-abd2-8b90524d9858
[2024-01-11T11:05:27.528Z] x-ms-version:2023-08-03
[2024-01-11T11:05:27.529Z] Date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.530Z] Connection:keep-alive
[2024-01-11T11:05:27.531Z] Keep-Alive:REDACTED
[2024-01-11T11:05:27.532Z] Last-Modified:Thu, 11 Jan 2024 11:03:51 GMT
[2024-01-11T11:05:27.533Z] Content-Length:0
[2024-01-11T11:05:27.534Z]
[2024-01-11T11:05:27.539Z] Request [05c13415-4d81-429d-a46a-693110c5c7c8] HEAD http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-hosts/locks/csjhbn89vzjg3-2090646957/host
[2024-01-11T11:05:27.540Z] x-ms-version:2021-08-06
[2024-01-11T11:05:27.541Z] Accept:application/xml
[2024-01-11T11:05:27.542Z] x-ms-client-request-id:05c13415-4d81-429d-a46a-693110c5c7c8
[2024-01-11T11:05:27.543Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:27.544Z] User-Agent:azsdk-net-Storage.Blobs/12.13.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:27.545Z] x-ms-date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.546Z] Authorization:REDACTED
[2024-01-11T11:05:27.546Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:27.587Z] Response [05c13415-4d81-429d-a46a-693110c5c7c8] 200 OK (00.0s)
[2024-01-11T11:05:27.588Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:27.590Z] x-ms-creation-time:Fri, 22 Dec 2023 10:17:04 GMT
[2024-01-11T11:05:27.590Z] x-ms-meta-FunctionInstance:REDACTED
[2024-01-11T11:05:27.591Z] x-ms-blob-type:BlockBlob
[2024-01-11T11:05:27.592Z] x-ms-lease-duration:fixed
[2024-01-11T11:05:27.593Z] x-ms-lease-state:leased
[2024-01-11T11:05:27.593Z] x-ms-lease-status:locked
[2024-01-11T11:05:27.594Z] ETag:"0x253ADA9DEF85160"
[2024-01-11T11:05:27.595Z] x-ms-client-request-id:05c13415-4d81-429d-a46a-693110c5c7c8
[2024-01-11T11:05:27.596Z] x-ms-request-id:6511ed54-fae6-4c49-85b9-66d6adf3b0f7
[2024-01-11T11:05:27.597Z] x-ms-version:2023-08-03
[2024-01-11T11:05:27.602Z] Date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.603Z] Accept-Ranges:bytes
[2024-01-11T11:05:27.604Z] x-ms-server-encrypted:true
[2024-01-11T11:05:27.605Z] x-ms-access-tier:Hot
[2024-01-11T11:05:27.606Z] x-ms-access-tier-inferred:true
[2024-01-11T11:05:27.607Z] x-ms-access-tier-change-time:Fri, 22 Dec 2023 10:17:04 GMT
[2024-01-11T11:05:27.608Z] Connection:keep-alive
[2024-01-11T11:05:27.609Z] Keep-Alive:REDACTED
[2024-01-11T11:05:27.609Z] Last-Modified:Thu, 11 Jan 2024 11:03:51 GMT
[2024-01-11T11:05:27.610Z] Content-Length:0
[2024-01-11T11:05:27.611Z] Content-Type:application/octet-stream
[2024-01-11T11:05:27.612Z] Content-MD5:1B2M2Y8AsgTpgAmY7PhCfg==
[2024-01-11T11:05:27.612Z]
[2024-01-11T11:05:27.627Z] Request [1f47b46c-b097-43ad-84e7-1187227ff5be] PUT http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-hosts/locks/csjhbn89vzjg3-2090646957/host?comp=metadata
[2024-01-11T11:05:27.630Z] x-ms-meta-FunctionInstance:REDACTED
[2024-01-11T11:05:27.631Z] x-ms-lease-id:000000000000000000000000CD5B5480
[2024-01-11T11:05:27.632Z] x-ms-version:2021-08-06
[2024-01-11T11:05:27.633Z] Accept:application/xml
[2024-01-11T11:05:27.634Z] x-ms-client-request-id:1f47b46c-b097-43ad-84e7-1187227ff5be
[2024-01-11T11:05:27.635Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:27.636Z] User-Agent:azsdk-net-Storage.Blobs/12.13.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:27.637Z] x-ms-date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.638Z] Authorization:REDACTED
[2024-01-11T11:05:27.639Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:27.681Z] Response [1f47b46c-b097-43ad-84e7-1187227ff5be] 200 OK (00.0s)
[2024-01-11T11:05:27.683Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:27.684Z] ETag:"0x224985FF23CAA40"
[2024-01-11T11:05:27.686Z] x-ms-client-request-id:1f47b46c-b097-43ad-84e7-1187227ff5be
[2024-01-11T11:05:27.686Z] x-ms-request-id:dc8a7da4-1e7c-4b9c-9fd5-adae91bea70e
[2024-01-11T11:05:27.687Z] x-ms-version:2023-08-03
[2024-01-11T11:05:27.688Z] Date:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.689Z] x-ms-request-server-encrypted:true
[2024-01-11T11:05:27.690Z] Connection:keep-alive
[2024-01-11T11:05:27.692Z] Keep-Alive:REDACTED
[2024-01-11T11:05:27.693Z] Last-Modified:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:27.693Z] Content-Length:0
[2024-01-11T11:05:27.694Z]
[2024-01-11T11:05:27.701Z] Host lock lease acquired by instance ID '000000000000000000000000CD5B5480'.
[2024-01-11T11:05:27.893Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] attempt number 2 took 01.1s
[2024-01-11T11:05:27.897Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
[2024-01-11T11:05:27.900Z] Metadata:REDACTED
[2024-01-11T11:05:27.902Z] x-ms-client-request-id:40f919aa-9f8c-4bb7-bae1-7be24d269181
[2024-01-11T11:05:27.903Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:27.904Z] User-Agent:azsdk-net-Identity/1.10.4 (.NET 6.0.24; Microsoft Windows 10.0.22621)[2024-01-11T11:05:27.905Z] client assembly: Azure.Identity
[2024-01-11T11:05:28.949Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] exception System.Threading.Tasks.TaskCanceledException: The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.
[2024-01-11T11:05:28.952Z]  ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
[2024-01-11T11:05:28.953Z]    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
[2024-01-11T11:05:28.954Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-01-11T11:05:28.958Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-01-11T11:05:28.959Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-01-11T11:05:28.960Z]    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:28.961Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:28.962Z]    --- End of inner exception stack trace ---
[2024-01-11T11:05:28.963Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)
[2024-01-11T11:05:28.964Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:28.965Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:32.543Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] attempt number 3 took 01.1s
[2024-01-11T11:05:32.551Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
[2024-01-11T11:05:32.553Z] Metadata:REDACTED
[2024-01-11T11:05:32.554Z] x-ms-client-request-id:40f919aa-9f8c-4bb7-bae1-7be24d269181
[2024-01-11T11:05:32.555Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:32.556Z] User-Agent:azsdk-net-Identity/1.10.4 (.NET 6.0.24; Microsoft Windows 10.0.22621)[2024-01-11T11:05:32.557Z] client assembly: Azure.Identity
[2024-01-11T11:05:33.576Z] Request [40f919aa-9f8c-4bb7-bae1-7be24d269181] exception System.Threading.Tasks.TaskCanceledException: The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.
[2024-01-11T11:05:33.577Z]  ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
[2024-01-11T11:05:33.578Z]    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
[2024-01-11T11:05:33.580Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-01-11T11:05:33.581Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-01-11T11:05:33.582Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-01-11T11:05:33.583Z]    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.584Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.585Z]    --- End of inner exception stack trace ---
[2024-01-11T11:05:33.586Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)
[2024-01-11T11:05:33.587Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.589Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.607Z] False MSAL 4.56.0.0 MSAL.NetCore .NET 6.0.24 Microsoft Windows 10.0.22621 [2024-01-11 11:05:33Z - c4cb8c80-5109-4966-8596-5859a79390a8] Exception type: Azure.Identity.CredentialUnavailableException
[2024-01-11T11:05:33.608Z] ---> Inner Exception Details
[2024-01-11T11:05:33.609Z] Exception type: System.AggregateException
[2024-01-11T11:05:33.610Z] ---> Inner Exception Details
[2024-01-11T11:05:33.611Z] Exception type: System.Threading.Tasks.TaskCanceledException
[2024-01-11T11:05:33.612Z] ---> Inner Exception Details
[2024-01-11T11:05:33.613Z] Exception type: System.Threading.Tasks.TaskCanceledException
[2024-01-11T11:05:33.614Z]
[2024-01-11T11:05:33.615Z]    at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
[2024-01-11T11:05:33.616Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-01-11T11:05:33.617Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-01-11T11:05:33.618Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-01-11T11:05:33.621Z]    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.623Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.624Z] === End of inner exception stack trace ===
[2024-01-11T11:05:33.625Z]
[2024-01-11T11:05:33.625Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ThrowIfCancellationRequestedOrTimeout(CancellationToken originalToken, CancellationToken timeoutToken, Exception inner, TimeSpan timeout)
[2024-01-11T11:05:33.627Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.628Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.629Z]    at Azure.Core.Pipeline.RedirectPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.631Z]    at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.632Z] === End of inner exception stack trace ===
[2024-01-11T11:05:33.633Z]
[2024-01-11T11:05:33.635Z]    at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-01-11T11:05:33.636Z]    at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
[2024-01-11T11:05:33.637Z]    at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
[2024-01-11T11:05:33.638Z] === End of inner exception stack trace ===
[2024-01-11T11:05:33.639Z]
[2024-01-11T11:05:33.640Z]    at Azure.Identity.ImdsManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
[2024-01-11T11:05:33.641Z]    at Azure.Identity.ManagedIdentityClient.AuthenticateCoreAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
[2024-01-11T11:05:33.642Z]    at Azure.Identity.ManagedIdentityClient.AppTokenProviderImpl(AppTokenProviderParameters parameters)
[2024-01-11T11:05:33.643Z]    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.SendTokenRequestToAppTokenProviderAsync(ILoggerAdapter logger, CancellationToken cancellationToken)
[2024-01-11T11:05:33.644Z]    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.GetAccessTokenAsync(CancellationToken cancellationToken, ILoggerAdapter logger)
[2024-01-11T11:05:33.647Z]    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.ExecuteAsync(CancellationToken cancellationToken)
[2024-01-11T11:05:33.648Z]    at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
[2024-01-11T11:05:33.657Z] ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:  Exception: Azure.Identity.CredentialUnavailableException (0x80131500): ManagedIdentityCredential authentication unavailable. Multiple attempts failed to obtain a token from the managed identity endpoint.
[2024-01-11T11:05:33.659Z]  ---> System.AggregateException (0x80131500): Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.) (The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.) (The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.) (The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.)
[2024-01-11T11:05:33.660Z]  ---> System.Threading.Tasks.TaskCanceledException (0x8013153b): The operation was cancelled because it exceeded the configured timeout of 0:00:01. Network timeout can be adjusted in ClientOptions.Retry.NetworkTimeout.
[2024-01-11T11:05:33.661Z]  ---> System.Threading.Tasks.TaskCanceledException (0x8013153b): A task was canceled.
[2024-01-11T11:05:33.669Z] VisualStudioCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:39.740Z] Request [0ada2945-0b45-4fa5-a22b-f65bc079f867] PUT http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-hosts/locks/csjhbn89vzjg3-2090646957/host?comp=lease
[2024-01-11T11:05:39.747Z] x-ms-lease-action:renew
[2024-01-11T11:05:39.749Z] x-ms-lease-id:000000000000000000000000CD5B5480
[2024-01-11T11:05:39.750Z] x-ms-version:2021-08-06
[2024-01-11T11:05:39.751Z] Accept:application/xml
[2024-01-11T11:05:39.752Z] x-ms-client-request-id:0ada2945-0b45-4fa5-a22b-f65bc079f867
[2024-01-11T11:05:39.753Z] x-ms-return-client-request-id:true
[2024-01-11T11:05:39.754Z] User-Agent:azsdk-net-Storage.Blobs/12.13.0 (.NET 6.0.24; Microsoft Windows 10.0.22621)
[2024-01-11T11:05:39.755Z] x-ms-date:Thu, 11 Jan 2024 11:05:39 GMT
[2024-01-11T11:05:39.756Z] Authorization:REDACTED
[2024-01-11T11:05:39.757Z] client assembly: Azure.Storage.Blobs
[2024-01-11T11:05:39.796Z] Response [0ada2945-0b45-4fa5-a22b-f65bc079f867] 200 OK (00.0s)
[2024-01-11T11:05:39.799Z] Server:Azurite-Blob/3.26.0
[2024-01-11T11:05:39.801Z] ETag:"0x224985FF23CAA40"
[2024-01-11T11:05:39.803Z] x-ms-lease-id:000000000000000000000000CD5B5480
[2024-01-11T11:05:39.806Z] x-ms-client-request-id:0ada2945-0b45-4fa5-a22b-f65bc079f867
[2024-01-11T11:05:39.807Z] x-ms-request-id:a3372dc5-9f54-40f9-8200-10baeeb4e79e
[2024-01-11T11:05:39.809Z] x-ms-version:2023-08-03
[2024-01-11T11:05:39.810Z] Date:Thu, 11 Jan 2024 11:05:39 GMT
[2024-01-11T11:05:39.811Z] Connection:keep-alive
[2024-01-11T11:05:39.812Z] Keep-Alive:REDACTED
[2024-01-11T11:05:39.813Z] Last-Modified:Thu, 11 Jan 2024 11:05:27 GMT
[2024-01-11T11:05:39.815Z] Content-Length:0
[2024-01-11T11:05:39.816Z]
[2024-01-11T11:05:40.170Z] VisualStudioCredential.GetToken succeeded. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:  ExpiresOn: 2024-01-11T12:05:05.0000000+00:00
[2024-01-11T11:05:40.178Z] DefaultAzureCredential credential selected: Azure.Identity.VisualStudioCredential
[2024-01-11T11:05:40.181Z] DefaultAzureCredential.GetToken succeeded. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:  ExpiresOn: 2024-01-11T12:05:05.0000000+00:00
[2024-01-11T11:05:40.183Z] VisualStudioCredential.GetToken invoked. Scopes: [ https://eventhubs.azure.net/.default ] ParentRequestId:
[2024-01-11T11:05:43.688Z] Completed retrieving properties for Event Hub: xxxxxxxxxxxx.
[2024-01-11T11:05:43.739Z] Request [dae24d15-58a5-4efb-8a12-3c340c0fc369] GET http://127.0.0.1:10000/devstoreaccount1/azure-webjobs-eventhub?restype=container&comp=list&prefix=xxxxxxxxxxxx.servicebus.windows.net%2Fxxxxxxxxxxxx%2Feme%2Fownership%2F&include=Metadata
whynot1597 commented 10 months ago

@christothes

I'm getting this error with the same setup as @BC89. Here is my full error.

Azure.Identity.AuthenticationFailedException
  HResult=0x80131500
  Message=ManagedIdentityCredential authentication failed: Service request failed.
Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable host.)

Headers:
Connection: close

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
  Source=Azure.Identity
  StackTrace:
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
   at Azure.Identity.ManagedIdentityCredential.<GetTokenImplAsync>d__16.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Identity.ManagedIdentityCredential.<GetTokenAsync>d__14.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Identity.DefaultAzureCredential.<GetTokenFromSourcesAsync>d__14.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Identity.DefaultAzureCredential.<GetTokenImplAsync>d__12.MoveNext()
   at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
   at Azure.Identity.DefaultAzureCredential.<GetTokenImplAsync>d__12.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Identity.DefaultAzureCredential.<GetTokenAsync>d__11.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.<GetHeaderValueFromCredentialAsync>d__9.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1.ConfiguredValueTaskAwaiter.GetResult()
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.<GetHeaderValueAsync>d__6.MoveNext()
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.<GetHeaderValueAsync>d__6.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.<AuthenticateAndAuthorizeRequestAsync>d__12.MoveNext()
   at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.<ProcessAsync>d__11.MoveNext()
   at Azure.Core.Pipeline.RedirectPolicy.<ProcessAsync>d__7.MoveNext()
   at Azure.Core.Pipeline.RetryPolicy.<ProcessAsync>d__5.MoveNext()
   at Azure.Core.Pipeline.RetryPolicy.<ProcessAsync>d__5.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.UserAgentHeaderPolicy.<ProcessAsync>d__3.MoveNext()
   at Azure.Core.PageableHelpers.PageableImplementation`1.<GetNextResponseAsync>d__20.MoveNext()
   at System.Threading.Tasks.ValueTask`1.get_Result()
   at Azure.Core.PageableHelpers.PageableImplementation`1.<GetAsyncEnumerator>d__15.MoveNext()
   at Azure.Core.PageableHelpers.PageableImplementation`1.<GetAsyncEnumerator>d__15.System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult(Int16 token)
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass31_2.<<LoadSelectedKeyValues>b__2>d.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass31_2.<<LoadSelectedKeyValues>b__2>d.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.TracingUtils.<CallWithRequestTracing>d__3.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<LoadSelectedKeyValues>d__31.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass30_0.<<InitializeAsync>b__0>d.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<>c__DisplayClass43_0.<<ExecuteWithFailOverPolicyAsync>b__0>d.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<ExecuteWithFailOverPolicyAsync>d__42`1.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<ExecuteWithFailOverPolicyAsync>d__43.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<InitializeAsync>d__30.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<TryInitializeAsync>d__29.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.<LoadAsync>d__28.MoveNext()
   at Microsoft.Extensions.Configuration.AzureAppConfiguration.AzureAppConfigurationProvider.Load()
   at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
   at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
   at <internal code>

  This exception was originally thrown at this call stack:
    [External Code]

Inner Exception 1:
RequestFailedException: Service request failed.
Status: 403 (connecting to 169.254.169.254:80: connecting to 169.254.169.254:80: dial tcp 169.254.169.254:80: connectex: A socket operation was attempted to an unreachable host.)

Headers:
Connection: close
christothes commented 9 months ago

Looks like this is due to the error message text changing slightly. Will look at a fix that matches the error message more loosely.