search

Azure / azure-sdk-for-net

This repository is for active development of the Azure SDK for .NET. For consumers of the SDK we recommend visiting our public developer docs at https://learn.microsoft.com/dotnet/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-net.
MIT License
5.49k stars 4.81k forks source link

[QUERY] DefaultIdentityNotDefined error response from attempt to get User-assigned MSI token from a scope #45547

Closed AdamL-Microsoft closed 1 month ago

AdamL-Microsoft commented 3 months ago

Library name and version

Azure.Identity 1.12.0

Query/Question

Having an issue with the Azure.Identity library while using the GetToken() method in a pretty basic program that is essentially:

using Azure.Core;
using Azure.Identity;

var scope = "api://AZURE_TENANT.onmicrosoft.com/AN_AZURE_API_NAME/.default";
var clientId = "<GUID_FOR_MSI_CLIENT_ID>";

var credential = new ManagedIdentityCredential(clientId: clientId);
var token = credential.GetToken(new TokenRequestContext([scope]));
Console.WriteLine(token.Token);
return 0;

The MSI is in the same tenant as the API its trying to auth against and has a role granted to it to access the Azure API scope specified. The app is also running on a VM that has permission to use the user-assigned MSI.

However the program fails and gets a 404 Not found when tried with a DefaultIdentityNotDefined error. The referenced troubleshooting link has no signs of this error or anything related as far as I can tell.

Any idea whats wrong or that I've missed setting here?

The full output error/stacktrace is:

2024-08-15 18:11:50.283779: Unhandled exception. Azure.Identity.AuthenticationFailedException: ManagedIdentityCredential authentication failed: DefaultIdentityNotDefined 
2024-08-15 18:11:50.283779: Status: 404 (Not Found)
2024-08-15 18:11:50.283779: ErrorCode: DefaultIdentityNotDefined
2024-08-15 18:11:50.283779: 
2024-08-15 18:11:50.283779: Content:
2024-08-15 18:11:50.283779: {
2024-08-15 18:11:50.283779:   "error": {
2024-08-15 18:11:50.283779:     "code": "DefaultIdentityNotDefined",
2024-08-15 18:11:50.283779:     "message": "DefaultIdentityNotDefined ",
2024-08-15 18:11:50.283779:     "details": []
2024-08-15 18:11:50.283779:   }
2024-08-15 18:11:50.283779: }
2024-08-15 18:11:50.283779: Headers:
2024-08-15 18:11:50.283779: Date: Thu, 15 Aug 2024 18:11:49 GMT
2024-08-15 18:11:50.283779: Server: Kestrel
2024-08-15 18:11:50.283779: Transfer-Encoding: chunked
2024-08-15 18:11:50.283779: x-ms-request-id: e8e1e82a-2190-43f4-87d6-dd2e7d0db3b4
2024-08-15 18:11:50.283779: x-ms-client-request-id: 37122100-2414-4d8e-9e7f-daf15891af22
2024-08-15 18:11:50.283779: Content-Type: application/json; charset=utf-8
2024-08-15 18:11:50.283779: 
2024-08-15 18:11:50.283779: See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
2024-08-15 18:11:50.283779:  ---> Azure.RequestFailedException: DefaultIdentityNotDefined 
2024-08-15 18:11:50.283779:    at Azure.Identity.ManagedIdentitySource.HandleResponseAsync(Boolean async, TokenRequestContext context, HttpMessage message, CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Azure.Identity.ManagedIdentitySource.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Azure.Identity.ManagedIdentityClient.AuthenticateCoreAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Azure.Identity.ManagedIdentityClient.AppTokenProviderImpl(AppTokenProviderParameters parameters)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.SendTokenRequestToAppTokenProviderAsync(ILoggerAdapter logger, CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.GetAccessTokenAsync(CancellationToken cancellationToken, ILoggerAdapter logger)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.ExecuteAsync(CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<<RunAsync>b__1>d.MoveNext()
2024-08-15 18:11:50.283779: --- End of stack trace from previous location ---
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Utils.StopwatchService.MeasureCodeBlockAsync(Func`1 codeBlock)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken)
2024-08-15 18:11:50.283779:    at Microsoft.Identity.Client.ApiConfig.Executors.ConfidentialClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenForClientParameters clientParameters, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Azure.Identity.AbstractAcquireTokenParameterBuilderExtensions.ExecuteAsync[T](AbstractAcquireTokenParameterBuilder`1 builder, Boolean async, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientCoreAsync(String[] scopes, String tenantId, String claims, Boolean enableCae, Boolean async, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Azure.Identity.MsalConfidentialClient.AcquireTokenForClientAsync(String[] scopes, String tenantId, String claims, Boolean enableCae, Boolean async, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Azure.Identity.ManagedIdentityClient.AuthenticateAsync(Boolean async, TokenRequestContext context, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Azure.Identity.ManagedIdentityCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    --- End of inner exception stack trace ---
2024-08-15 18:11:50.306571:    at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex, String additionalMessage, Boolean isCredentialUnavailable)
2024-08-15 18:11:50.306571:    at Azure.Core.Pipeline.TaskExtensions.EnsureCompleted[T](ValueTask`1 task)
2024-08-15 18:11:50.306571:    at Azure.Identity.ManagedIdentityCredential.GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
2024-08-15 18:11:50.306571:    at Program.<Main>$(String[] args) in C:\Users\name\source\repos\project\src\msi-helper\Program.cs:line 10

Environment

Running on a Windows Server LTSC 2022 VM, with .Net 4.8, .Net core 8.0.4 (runtime) installed, called from powershell core 7.4.2

compiled for <TargetFramework>net8.0</TargetFramework>

(this is an EV2 agent in the Azure Production Management tenant)

github-actions[bot] commented 3 months ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

christothes commented 3 months ago

Hi @AdamL-Microsoft - Can you confirm that you have a user-assigned managed identity configured on the VM as described here?

github-actions[bot] commented 3 months ago

Hi @AdamL-Microsoft. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

AdamL-Microsoft commented 3 months ago

Hi @christothes,

I believe the MSI should be assigned since we're able to use the MSI with az CLI commands inside the instance, on review this is actually running on an Azure container instance we don't have direct ownership of. This error above is from an EV2 agent log output during a rollout.

We're thinking this might be a limitation of EV2 if this process is supported in vanilla Windows-based Azure Containers. Our first tests running the app on a VM with the MSI added to it were able to get a token without issue.

ostojicb commented 3 months ago

@christothes We're also experiencing the same issue with ACI

Container has UAMI assigned, and we're able to CURL the %IDENTITY_ENDPOINT% (not 169.254.169.254) from the CMD

However, Azure.Identity SDK is struggling to figure out the correct flow

christothes commented 3 months ago

Hi @AdamL-Microsoft - Could you verify that the endpoint responds if you curl the endpoint as explained here?

If that works, please capture some logging and we can see what is different about the two requests - details here

github-actions[bot] commented 3 months ago

Hi @AdamL-Microsoft. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

github-actions[bot] commented 2 months ago

Hi @AdamL-Microsoft, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

melliott-whitebox commented 2 months ago

I have the exact same error when attempting to access an Azure Key Vault from a C# application within a Windows-based Azure Container Instance. Both user- and system-assigned managed identities throw this error back, and there is nothing helpful in the documentation about this. Happy to provide more information because I am struggling to do authentication the Right Way and from what I can tell managed identities in windows-based containers simply don't work.

christothes commented 2 months ago

@melliott-whitebox - If you could provide the info described in this previous comment, we could troubleshoot further.

christothes commented 2 months ago

May be related to https://github.com/Azure/azure-sdk-for-net/issues/43076

melliott-whitebox commented 2 months ago

@melliott-whitebox - If you could provide the info described in this previous comment, we could troubleshoot further.

I've created a test application that does the following:

static void Main(string[] args)
{
    Console.WriteLine("Starting up");
    using AzureEventSourceListener listener = AzureEventSourceListener.CreateConsoleLogger(EventLevel.LogAlways);

    var managedClientId = "3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425";  // REDACTED
    Console.WriteLine($"About to access the keyvault using clientID: {managedClientId}");

    try
    {
        var scope = $"{managedClientId}/.default";

        var credential = new ManagedIdentityCredential(clientId: managedClientId);
        var token = credential.GetToken(new TokenRequestContext(new string[] { scope }));
        Console.WriteLine(token.Token);
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error getting token: " + ex.Message);
    }

    Console.WriteLine("COMPLETE");
    Task.Delay(5000).Wait();  // so I can get the container instance log 
}

This code has been deployed into ACR, and from there into ACI with an assigned identity. The identity is clearly assigned, based on the results of az container show:

  "identity": {
    "principalId": null,
    "tenantId": "5e1a2f36-XXXX-XXXX-XXXX-d009ae350bdc",
    "type": "UserAssigned",
    "userAssignedIdentities": {
      "/subscriptions/9ecab35c-XXXX-XXXX-XXXX-29cb4df1a04c/resourcegroups/WHALC1-Stream/providers/Microsoft.ManagedIdentity/userAssignedIdentities/whalc1-eh-core-reader": {
        "clientId": "3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425",
        "principalId": "63217ae1-XXXX-XXXX-XXXX-bafadfeee555"
      }
    }
  }

When the container instance runs, I get the following log. Same message, default identity cannot be found. 

Starting up
About to access the keyvault using clientID: 3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425
[Informational] Azure-Identity: ManagedIdentityCredential.GetToken invoked. Scopes: [ 3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425/.default ] ParentRequestId: 
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] MSAL MSAL.Desktop with assembly version '4.61.3.0'. CorrelationId(ca1114f9-43a9-4bf2-a7a3-daf68af05a5f)
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] === AcquireTokenForClientParameters ===
SendX5C: False
ForceRefresh: False

[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] 
=== Request Data ===
Authority Provided? - True
Scopes - 3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425/.default
Extra Query Params Keys (space separated) - 
ApiId - AcquireTokenForClient
IsConfidentialClient - True
SendX5C - False
LoginHint ? False
IsBrokerConfigured - False
HomeAccountId - False
CorrelationId - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f
UserAssertion set: False
LongRunningOboCacheKey set: False
Region configured: 

[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] === Token Acquisition (ClientCredentialRequest) started:
     Scopes: 3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425/.default
    Authority Host: login.microsoftonline.com
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] [Region discovery] Not using a regional authority. 
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] [Instance Discovery] Skipping Instance discovery because it is disabled. 
[Informational] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:39Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] [ClientCredentialRequest] Acquiring a token from the token provider.
[Warning] Azure-Identity: Service Fabric user assigned managed identity ClientId or ResourceId is not configurable at runtime.
[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] GET http://10.92.0.21:2377/metadata/identity/oauth2/token?api-version=1.0&api-version=2019-07-01-preview&resource=REDACTED&client_id=REDACTED
secret:REDACTED
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.12.0 (.NET Framework 4.8.4749.0; Microsoft Windows 10.0.17763 )
client assembly: Azure.Identity
[Warning] Azure-Core: Error response [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] 404 Not Found (00.1s)
Transfer-Encoding:chunked
x-ms-request-id:2cf132b9-de07-449a-bd76-cc8cb9b17703
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date:Wed, 04 Sep 2024 17:39:38 GMT
Server:Kestrel
Content-Type:application/json; charset=utf-8

[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] attempt number 1 took 00.2s
[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] GET http://10.92.0.21:2377/metadata/identity/oauth2/token?api-version=1.0&api-version=2019-07-01-preview&resource=REDACTED&client_id=REDACTED
secret:REDACTED
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.12.0 (.NET Framework 4.8.4749.0; Microsoft Windows 10.0.17763 )
client assembly: Azure.Identity
[Warning] Azure-Core: Error response [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] 404 Not Found (00.0s)
Transfer-Encoding:chunked
x-ms-request-id:1bec1cb2-70c1-4904-993f-c7dc5abe2d85
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date:Wed, 04 Sep 2024 17:39:39 GMT
Server:Kestrel
Content-Type:application/json; charset=utf-8

[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] attempt number 2 took 00.0s
[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] GET http://10.92.0.21:2377/metadata/identity/oauth2/token?api-version=1.0&api-version=2019-07-01-preview&resource=REDACTED&client_id=REDACTED
secret:REDACTED
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.12.0 (.NET Framework 4.8.4749.0; Microsoft Windows 10.0.17763 )
client assembly: Azure.Identity
[Warning] Azure-Core: Error response [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] 404 Not Found (00.0s)
Transfer-Encoding:chunked
x-ms-request-id:c6e4ee9b-2ba0-4d66-96f3-5be1b42a557f
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date:Wed, 04 Sep 2024 17:39:41 GMT
Server:Kestrel
Content-Type:application/json; charset=utf-8

[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] attempt number 3 took 00.0s
[Informational] Azure-Core: Request [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] GET http://10.92.0.21:2377/metadata/identity/oauth2/token?api-version=1.0&api-version=2019-07-01-preview&resource=REDACTED&client_id=REDACTED
secret:REDACTED
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
x-ms-return-client-request-id:true
User-Agent:azsdk-net-Identity/1.12.0 (.NET Framework 4.8.4749.0; Microsoft Windows 10.0.17763 )
client assembly: Azure.Identity
[Warning] Azure-Core: Error response [4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9] 404 Not Found (00.0s)
Transfer-Encoding:chunked
x-ms-request-id:8a836030-dbee-45fb-8a7e-c07d1b207d0c
x-ms-client-request-id:4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date:Wed, 04 Sep 2024 17:39:43 GMT
Server:Kestrel
Content-Type:application/json; charset=utf-8

[Error] Azure-Identity: False MSAL 4.61.3.0 MSAL.Desktop 4.8 or later Windows Server 2019 Datacenter [2024-09-04 17:39:44Z - ca1114f9-43a9-4bf2-a7a3-daf68af05a5f] Exception type: Azure.RequestFailedException
To see full exception details, enable PII Logging. See https://aka.ms/msal-net-logging
   at Azure.Identity.ManagedIdentitySource.<HandleResponseAsync>d__11.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Identity.ManagedIdentitySource.<AuthenticateAsync>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Identity.ManagedIdentityClient.<AuthenticateCoreAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Identity.ManagedIdentityClient.<AppTokenProviderImpl>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<SendTokenRequestToAppTokenProviderAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<GetAccessTokenAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.ClientCredentialRequest.<ExecuteAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<>c__DisplayClass11_1.<<RunAsync>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Utils.StopwatchService.<MeasureCodeBlockAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<RunAsync>d__11.MoveNext()

[Informational] Azure-Identity: ManagedIdentityCredential.GetToken was unable to retrieve an access token. Scopes: [ 3c0e7c15-XXXX-XXXX-XXXX-d43e9e283425/.default ] ParentRequestId:  Exception: Azure.Identity.AuthenticationFailedException (0x80131500): ManagedIdentityCredential authentication failed: DefaultIdentityNotDefined 
Status: 404 (Not Found)
ErrorCode: DefaultIdentityNotDefined

Content:
{
  "error": {
    "code": "DefaultIdentityNotDefined",
    "message": "DefaultIdentityNotDefined ",
    "details": []
  }
}

Headers:
Transfer-Encoding: chunked
x-ms-request-id: 8a836030-dbee-45fb-8a7e-c07d1b207d0c
x-ms-client-request-id: 4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date: Wed, 04 Sep 2024 17:39:43 GMT
Server: Kestrel
Content-Type: application/json; charset=utf-8

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
 ---> Azure.RequestFailedException (0x80131500): DefaultIdentityNotDefined 
Status: 404 (Not Found)
ErrorCode: DefaultIdentityNotDefined

Content:
{
  "error": {
    "code": "DefaultIdentityNotDefined",
    "message": "DefaultIdentityNotDefined ",
    "details": []
  }
}

Headers:
Transfer-Encoding: chunked
x-ms-request-id: 8a836030-dbee-45fb-8a7e-c07d1b207d0c
x-ms-client-request-id: 4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date: Wed, 04 Sep 2024 17:39:43 GMT
Server: Kestrel
Content-Type: application/json; charset=utf-8

Error getting token: ManagedIdentityCredential authentication failed: DefaultIdentityNotDefined 
Status: 404 (Not Found)
ErrorCode: DefaultIdentityNotDefined

Content:
{
  "error": {
    "code": "DefaultIdentityNotDefined",
    "message": "DefaultIdentityNotDefined ",
    "details": []
  }
}

Headers:
Transfer-Encoding: chunked
x-ms-request-id: 8a836030-dbee-45fb-8a7e-c07d1b207d0c
x-ms-client-request-id: 4ae1a3c6-1a39-4f61-8f1e-5c45ccead2c9
Date: Wed, 04 Sep 2024 17:39:43 GMT
Server: Kestrel
Content-Type: application/json; charset=utf-8

See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot
COMPLETED
melliott-whitebox commented 2 months ago

@christothes is this helpful at all?

https://github.com/Azure/azure-sdk-for-net/issues/45547#issuecomment-2329663289

christothes commented 2 months ago

Hi @melliott-whitebox Sorry for the delay!

Looking at your log output, it appears that it's attempting to reach an AKS pod identity (based on the url http://10.92.0.21:2377/metadata/identity/oauth2/token) rather than the ACI managed identity which should be at http://169.254.169.254/metadata/identity/oauth2/token).

Is the code running from the ACI? Can you validate that you can receive a token via the bash steps mentioned here?

melliott-whitebox commented 2 months ago

@christothes it's definitely running within ACI. I'm not sure I can run a bash shell within the container, since it is a windows container. is this possible?

christothes commented 2 months ago

@christothes it's definitely running within ACI. I'm not sure I can run a bash shell within the container, since it is a windows container. is this possible?

Yes - it appears to be via the commands in the article - https://learn.microsoft.com/en-us/azure/container-instances/container-instances-managed-identity#use-user-assigned-identity-to-get-secret-from-key-vault

github-actions[bot] commented 2 months ago

Hi @AdamL-Microsoft, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

melliott-whitebox commented 2 months ago

I was able to get this working using cmd.exe as bash shell is not available within windows containers.

One thing I note from that article is that it explicitly states "For Windows containers, metadata server (169.254.169.254) isn't available." This is confirmed by the pre-set env var %IDENTITY_ENDPOINT% referring to a 10.92.0.19 URL.

The other main difference I'll note is that the shell script requests a token using the managed identity's principal id, whereas the code sample I provided works with the client id.

commands and output below. 

Microsoft Windows [Version 10.0.17763.6189]
(c) 2018 Microsoft Corporation. All rights reserved.

C:\app>echo %IDENTITY_ENDPOINT% 
http://10.92.0.19:2377/metadata/identity/oauth2/token?api-version=1.0 

C:\app>echo %IDENTITY_HEADER% 
<REDACTED-SECRET>

C:\app>curl -G -v %IDENTITY_ENDPOINT% --data-urlencode resource=https://vault.azure.net --data-urlencode principalId=<REDACTED-PRINCIPAL-ID> -H secret:%IDENTITY_HEADER%
*   Trying 10.92.0.19:2377... 
* Connected to 10.92.0.19 (10.92.0.19) port 2377
> GET /metadata/identity/oauth2/token?api-version=1.0&resource=https%3a%2f%2fvault.azure.net&principalId=63217ae1-0e4b
-4e37-a7e3-bafadfeee555 HTTP/1.1
> Host: 10.92.0.19:2377
> User-Agent: curl/8.7.1
> Accept: */*
> secret:<REDACTED-SECRET>
>
* Request completely sent off
< HTTP/1.1 200 OK 
< Date: Wed, 18 Sep 2024 17:26:40 GMT
< Content-Type: application/json; charset=utf-8
< Server: Kestrel
< Transfer-Encoding: chunked
< x-ms-request-id: d835fefe-332f-49cc-a6b9-8869f16a5f88
<
{
  "token_type": "Bearer",
  "expires_on": "1726765899",
  "access_token": "<REDACTED-TOKEN>",
  "resource": "https://vault.azure.net"
}* Connection #0 to host 10.92.0.19 left intact

C:\app>set TOKEN=<REDACTED-TOKEN>

C:\app>curl https://<REDACTED-KEYVAULT>.vault.azure.net/secrets/ConnectionString/?api-version=7.4 -H "Authorization: Bearer
 %TOKEN%"
{"value":"<REDACTED>","id":"https://<REDACTED-KEYVAULT>.vault.azure.net/secrets/ConnectionString/<REDACTED>","attributes":{"enabled":true,"created":1719873974,"updated":1719873974,"recoveryLevel":"Recoverable+Purgeable","recoverableDays":90},"tags":{}}
github-actions[bot] commented 1 month ago

Hi @AdamL-Microsoft, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!

christothes commented 1 month ago

Thanks @melliott-whitebox Thanks - we didn't specifically add support for ACI scenarios, we are just falling back to the standard IMDS endpoint logic here. It looks like we'll need to make some changes to accommodate this scenario.

github-actions[bot] commented 1 month ago

Hi @AdamL-Microsoft, we're sending this friendly reminder because we haven't heard back from you in 7 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 14 days of this comment the issue will be automatically closed. Thank you!