search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.23k stars 4.57k forks source link

[BUG] DefaultAzureCredential fails to pick up Managed Identity #45014

Closed dozouf closed 1 month ago

dozouf commented 1 month ago

Library name and version

Azure.Identity 1.12.0

Describe the bug

My team develops on a shared Azure WVD. When my coworker debugs our application in Visual Studio 2022, everything works fine. When I try to debug the application, a CredentialUnavailableException is thrown in System.Threading.Tasks.Task<TResult>.InnerInvoke() (Future.cs) with the message, "EnvironmentCredential authentication unavailable. Environment variables are not fully configured," when obtaining a DefaultAzureCredential which is expected to pick up the WVD's Managed Identity.

If I specifically exclude Environment credentials, the same exception will be thrown with the message, "WorkloadIdentityCredential authentication unavailable. The workload options are not fully configured."

If I exclude workload credentials, as well, an ImdsManagedIdentitySource.ProbeRequestResponseException is thrown in System.Threading.Tasks.ValueTask<TResult>.Result (ValueTask.cs) with a message simply restating the exception that was thrown.

My coworker has confirmed the Managed Identity credential is working for him by excluding all of the other credentials when he debugs the application on the same WVD.

Expected behavior

DefaultAzureCredential picks up the WVD's Managed Identity.

Actual behavior

A CredentialUnavailableException is thrown with the message "Environment variables aren't fully configured."

Reproduction Steps

// In Startup.cs
var azureDefaultCredential = new DefaultAzureCredential();
services.AddSingleton(azureDefaultCredential);

Environment

Azure Windows Virtual Desktop Visual Studio 2022 (17.10.3)

Azure.Identity 1.12.0

github-actions[bot] commented 1 month ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

dozouf commented 1 month ago

I was able to boil this down to a simple project that still demonstrates the issue for me. I just created a new Blazor Web App and installed NuGet packages for Azure.Identity 1.12.0 and Azure.Extensions.AspNetCore.Configuration.Secrets 1.3.1. I added the following to my appsettings.json:

{ "Endpoints": { "KeyVault": "https://some-vault.vault.azure.net/" } }

And then updated Program.cs to:

using Azure.Core.Diagnostics;
using Azure.Identity;
using BlazorApp1.Components;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container.
builder.Services.AddRazorComponents()
    .AddInteractiveServerComponents();

var app = builder
  .Build();

using var listener = AzureEventSourceListener.CreateConsoleLogger();

var vaultEndpoint = builder.Configuration.GetSection("Endpoints")?["KeyVault"];

if (!string.IsNullOrWhiteSpace(vaultEndpoint))
  builder.Configuration.AddAzureKeyVault(
    new Uri(vaultEndpoint),
    new DefaultAzureCredential(
      new DefaultAzureCredentialOptions
      {
        Diagnostics =
        {
          LoggedHeaderNames = {"x-ms-request-id"},
          LoggedQueryParameters = {"api-version"},
          IsLoggingContentEnabled = true
        },
        //ExcludeEnvironmentCredential = true,
        //ExcludeWorkloadIdentityCredential = true
      }
    )
  );

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
  app.UseDeveloperExceptionPage();
}
else
{
  app.UseExceptionHandler("/Error", createScopeForErrors: true);
  // The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
  app.UseHsts();
}

app.UseHttpsRedirection();

app.UseStaticFiles();
app.UseAntiforgery();

app.MapRazorComponents<App>()
    .AddInteractiveServerRenderMode();

app.Run();

When I Debug the app as listed above, I get a CredentialUnavailableException in Azure.Extensions.AspNetCore.Configuration.Secrets.AzureKeyVaultConfigurationProvider.Load() with the message, EnvironmentCredential authentication unavailable. Environment variables are not fully configured, and the following is output to the console:

[Informational] Azure-Core: Request [a2502fc1-3634-4773-b464-a31b71fda14b] GET https://asc-dataops-vault-dev.vault.azure.net/secrets/?api-version=7.2
Content-Type:application/json
Accept:application/json
x-ms-client-request-id:a2502fc1-3634-4773-b464-a31b71fda14b
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Security.KeyVault.Secrets/4.2.0 (.NET 8.0.6; Microsoft Windows 10.0.19045)
client assembly: Azure.Security.KeyVault.Secrets
[Warning] Azure-Core: Error response [a2502fc1-3634-4773-b464-a31b71fda14b] 401 Unauthorized (00.7s)
Cache-Control:no-cache
Pragma:no-cache
WWW-Authenticate:Bearer authorization="https://login.microsoftonline.com/85fedf24-5af7-459c-b935-b978cf6c8ca0", resource="https://vault.azure.net"
x-ms-keyvault-region:eastus
x-ms-client-request-id:a2502fc1-3634-4773-b464-a31b71fda14b
x-ms-request-id:29272384-2f49-42d4-ab32-2efbfa970456
x-ms-keyvault-service-version:1.9.1625.1
x-ms-keyvault-network-info:conn_type=PrivateLink;private_endpoint=/subscriptions/bc69800c-2278-4fe4-b59e-3220716ea5cc/resourceGroups/RG_Databases_Dev/providers/Microsoft.Network/privateEndpoints/plink-vault-dataops-dev;addr=10.64.33.11;act_addr_fam=InterNetworkV6;
X-Content-Type-Options:REDACTED
Strict-Transport-Security:REDACTED
Date:Tue, 16 Jul 2024 14:17:04 GMT
Content-Length:97
Content-Type:application/json; charset=utf-8
Expires:-1

[Informational] Azure-Identity: DefaultAzureCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: a2502fc1-3634-4773-b464-a31b71fda14b
[Informational] Azure-Identity: EnvironmentCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: a2502fc1-3634-4773-b464-a31b71fda14b
[Informational] Azure-Identity: EnvironmentCredential.GetToken was unable to retrieve an access token. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: a2502fc1-3634-4773-b464-a31b71fda14b Exception: Azure.Identity.CredentialUnavailableException (0x80131500): EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot

When I uncomment the lines excluding EnvironmentCredentials and WorkloadIdentityCredentials, I get a ProbeRequestResponseException in System.Threading.Tasks.ValueTask<TResult>.Result with the message, Exception of type 'Azure.Identity.ImdsManagedIdentitySource+ProbeRequestResponseException' was thrown, and the following is output to the console:

[Informational] Azure-Core: Request [c09d43d2-1a6f-4be7-b765-eb2ca27982c7] GET https://asc-dataops-vault-dev.vault.azure.net/secrets/?api-version=7.2
Content-Type:application/json
Accept:application/json
x-ms-client-request-id:c09d43d2-1a6f-4be7-b765-eb2ca27982c7
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Security.KeyVault.Secrets/4.2.0 (.NET 8.0.6; Microsoft Windows 10.0.19045)
client assembly: Azure.Security.KeyVault.Secrets
[Warning] Azure-Core: Error response [c09d43d2-1a6f-4be7-b765-eb2ca27982c7] 401 Unauthorized (00.3s)
Cache-Control:no-cache
Pragma:no-cache
WWW-Authenticate:Bearer authorization="https://login.microsoftonline.com/85fedf24-5af7-459c-b935-b978cf6c8ca0", resource="https://vault.azure.net"
x-ms-keyvault-region:eastus
x-ms-client-request-id:c09d43d2-1a6f-4be7-b765-eb2ca27982c7
x-ms-request-id:97de2db5-751f-4f91-be32-4b291bd2af42
x-ms-keyvault-service-version:1.9.1625.1
x-ms-keyvault-network-info:conn_type=PrivateLink;private_endpoint=/subscriptions/bc69800c-2278-4fe4-b59e-3220716ea5cc/resourceGroups/RG_Databases_Dev/providers/Microsoft.Network/privateEndpoints/plink-vault-dataops-dev;addr=10.64.33.11;act_addr_fam=InterNetworkV6;
X-Content-Type-Options:REDACTED
Strict-Transport-Security:REDACTED
Date:Tue, 16 Jul 2024 14:23:45 GMT
Content-Length:97
Content-Type:application/json; charset=utf-8
Expires:-1

[Informational] Azure-Identity: DefaultAzureCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: c09d43d2-1a6f-4be7-b765-eb2ca27982c7
[Informational] Azure-Identity: ManagedIdentityCredential.GetToken invoked. Scopes: [ https://vault.azure.net/.default ] ParentRequestId: c09d43d2-1a6f-4be7-b765-eb2ca27982c7
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] MSAL MSAL.NetCore with assembly version '4.61.3.0'. CorrelationId(d0aec8e7-eee7-4dd8-80a1-a23b47a30311)
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] === AcquireTokenForClientParameters ===
SendX5C: False
ForceRefresh: False

[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311]
=== Request Data ===
Authority Provided? - True
Scopes - https://vault.azure.net/.default
Extra Query Params Keys (space separated) -
ApiId - AcquireTokenForClient
IsConfidentialClient - True
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - d0aec8e7-eee7-4dd8-80a1-a23b47a30311
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured:

[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] === Token Acquisition (ClientCredentialRequest) started:
         Scopes: https://vault.azure.net/.default
        Authority Host: login.microsoftonline.com
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] [Region discovery] Not using a regional authority.
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] [Instance Discovery] Skipping Instance discovery because it is disabled.
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.NetCore .NET 8.0.6 Microsoft Windows 10.0.19045 [2024-07-16 14:23:46Z - d0aec8e7-eee7-4dd8-80a1-a23b47a30311] [ClientCredentialRequest] Acquiring a token from the token provider.
[Informational] Azure-Core: Request [c5417ca4-4f7a-4122-818b-e9ef05494731] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED
x-ms-client-request-id:c5417ca4-4f7a-4122-818b-e9ef05494731
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.12.0 (.NET 8.0.6; Microsoft Windows 10.0.19045)
client assembly: Azure.Identity
[Warning] Azure-Core: Error response [c5417ca4-4f7a-4122-818b-e9ef05494731] 400 Bad Request (00.1s)
Server:IMDS/150.870.65.1324
x-ms-request-id:2f899709-350d-4b36-bd2f-92c52852ac93
Date:Tue, 16 Jul 2024 14:23:46 GMT
Content-Type:application/json; charset=utf-8
Content-Length:88

[Informational] Azure-Core: Error response [c5417ca4-4f7a-4122-818b-e9ef05494731] content: {"error":"invalid_request","error_description":"Required metadata header not specified"}
christothes commented 1 month ago

Hi @dozouf - It appears that you are looking at first chance exceptions during the DefaultAzureCredential credential selection process. The errors you mention will always be logged while the credential iterates through each credential type until it finds one that works, or all of them fail. See this documentation for details.

Would you mind providing the logging output after reproducing this with logging enabled? Could you also enable logging for the working scenario so that we can see which credential is being successfully selected in that case also? My guess is that it is not managed identity credential.

github-actions[bot] commented 1 month ago

Hi @dozouf. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

dozouf commented 1 month ago

I just updated the thread this morning with the output from the logging in the failing scenarios. I'll need to wait for my coworker to have time to run the project to capture the logging from the working scenario. He's out of the office today so I'll try to get it tomorrow. I might just be confused because the debugging is stopping on these expected exceptions for me but not for him. Is there some setting in VS to ignore these expected exceptions? We didn't notice any obviously significant differences in his General Debugging settings versus mine.

christothes commented 1 month ago

Perhaps your exception settings are different. Have you tried configuring it to continue on user-unhandled exceptions? https://learn.microsoft.com/en-us/visualstudio/debugger/managing-exceptions-with-the-debugger?view=vs-2022#BKMK_UserUnhandled

The last exception you show in your logging is also handled internally, and there should be a subsequent request made to the managed identity endpoint to fetch the token: [Informational] Azure-Core: Error response [c5417ca4-4f7a-4122-818b-e9ef05494731] content: {"error":"invalid_request","error_description":"Required metadata header not specified"}

github-actions[bot] commented 1 month ago

Hi @dozouf. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

dozouf commented 1 month ago

Ok, we found that disabling Enable Just My Code in the General Debugging options solved the issue. This seems counterintuitive, but I'll chalk it up to my lack of a deep understanding of ASP.NET error handling. Thanks for the last link above; that provided the solution.