[Bug] Custom api scopes not working with WAM

jurgenrapp commented 9 months ago

Library version used

4.58.1

.NET version

.NET 6.0

Scenario

PublicClient - desktop app

Is this a new or an existing app?

This is a new app or experiment

Issue description and reproduction steps

Unable to request custom api scopes when .WithBroker (WAM) is enabled, graph scopes like "user.read" works so the error code indicating that the redirect uri is not configured on the app is not correct.

Test case 2 in the code snippets fails: Exception thrown: 'Microsoft.Identity.Client.MsalServiceException' in System.Private.CoreLib.dll MSAL.NetCore.4.58.1.0.MsalServiceException: ErrorCode: WAM_provider_error_3399614466 Microsoft.Identity.Client.MsalServiceException: WAM Error
Error Code: 3399614466 Error Message: IncorrectConfiguration WAM Error Message: (pii) Internal Error Code: 557973643 Possible causes:

Invalid redirect uri - ensure you have configured the following url in the application registration in Azure Portal: ms-appx-web://microsoft.aad.brokerplugin/

at Microsoft.Identity.Client.Platforms.Features.RuntimeBroker.WamAdapters.HandleResponse(AuthResult authResult, AuthenticationRequestParameters authenticationRequestParameters, ILoggerAdapter logger, String errorMessage) at Microsoft.Identity.Client.Platforms.Features.RuntimeBroker.RuntimeBroker.SignInInteractivelyAsync(AuthenticationRequestParameters authenticationRequestParameters, AcquireTokenInteractiveParameters acquireTokenInteractiveParameters) at Microsoft.Identity.Client.Platforms.Features.RuntimeBroker.RuntimeBroker.AcquireTokenInteractiveAsync(AuthenticationRequestParameters authenticationRequestParameters, AcquireTokenInteractiveParameters acquireTokenInteractiveParameters) at Microsoft.Identity.Client.Internal.Broker.BrokerInteractiveRequestComponent.FetchTokensAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.FetchTokensFromBrokerAsync(String brokerInstallUrl, CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.GetTokenResponseAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.InteractiveRequest.ExecuteAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.Internal.Requests.RequestBase.RunAsync(CancellationToken cancellationToken) at Microsoft.Identity.Client.ApiConfig.Executors.PublicClientExecutor.ExecuteAsync(AcquireTokenCommonParameters commonParameters, AcquireTokenInteractiveParameters interactiveParameters, CancellationToken cancellationToken) at WpfApp7.MainWindow.ButtonBase_OnClick(Object sender, RoutedEventArgs e) in MainWindow.xaml.cs:line 169 StatusCode: 0 ResponseBody:
Headers:

Relevant code snippets

BrokerOptions options = new BrokerOptions(BrokerOptions.OperatingSystems.Windows);
options.ListOperatingSystemAccounts = true;

var appWithBroker = PublicClientApplicationBuilder.Create(clientId)
    .WithBroker(options)
    .WithDefaultRedirectUri()
    .WithParentActivityOrWindow(GetWindowHandle)
    .Build();

var appNoBroker = PublicClientApplicationBuilder.Create(clientId)
    .WithDefaultRedirectUri()
    .WithParentActivityOrWindow(GetWindowHandle)
    .Build();

AuthenticationResult authResult;
var workingScope = new[] { "https://graph.microsoft.com/.default" };
var customApiScope = new[] { $"api://{clientId}/name.Backend" };

try
{
    // Test with app with .WithBroker
    authResult = await appWithBroker.AcquireTokenInteractive(workingScope).ExecuteAsync();
    Debug.WriteLine("AppWithBroker: workingScope: Scopes=" + string.Join(',', authResult.Scopes));

    authResult = await appWithBroker.AcquireTokenInteractive(customApiScope).ExecuteAsync();
    Debug.WriteLine("AppWithBroker: customApiScope: Scopes=" + string.Join(',', authResult.Scopes));
}
catch (Exception exception)
{
    Debug.WriteLine(exception);
}

// Test with app without .WithBroker
authResult = await appNoBroker.AcquireTokenInteractive(workingScope).ExecuteAsync();
Debug.WriteLine("AppNoBroker: workingScope: Scopes=" + string.Join(',', authResult.Scopes));

authResult = await appNoBroker.AcquireTokenInteractive(customApiScope).ExecuteAsync();
Debug.WriteLine("AppNoBroker: customApiScope: Scopes=" + string.Join(',', authResult.Scopes));

Expected behavior

All four test cases should return the scope(s) requested

AppWithBroker: workingScope: Scopes=profile,openid,https://graph.microsoft.com/.default,email,https://graph.microsoft.com/Contacts.Read,https://graph.microsoft.com/Contacts.ReadWrite,https://graph.microsoft.com/User.Read

AppWithBroker: customApiScope: Scopes=api:///name.Backend

AppNoBroker: workingScope: Scopes=profile,openid,email,https://graph.microsoft.com/Contacts.Read,https://graph.microsoft.com/Contacts.ReadWrite,https://graph.microsoft.com/User.Read,https://graph.microsoft.com/.default

AppNoBroker: customApiScope: Scopes=api:///name.Backend

Identity provider

Microsoft Entra ID (Work and School accounts and Personal Microsoft accounts)

Regression

No response

Solution and workarounds

No response

bgavrilMS commented 9 months ago

There are multiple ways to represent an app ID URI and I think WAM doesn't support them all due to the dependnecy on v1 endpoint.

@jurgenrapp - could you try to use a different app ID as per https://learn.microsoft.com/en-us/entra/identity-platform/reference-app-manifest#identifieruris-attribute

jurgenrapp commented 8 months ago

@bgavrilMS I have now tested to change the Expose an API/Application ID URI to all flavors below with same result they all work when .WithBroker is not set, but fails when .WithBroker is enabled: api://<appId> api://<tenantId>/<appId> api://<tenantId>/<string> api://<string>/<appId> https://<tenantInitialDomain>.onmicrosoft.com/<string>

I noticed that there seems to be a possibility to use ms-appx for the Application ID URI as well but I couldn't get it to accept the uri: ms-appx-web://<domain>.onmicrosoft.com/api

"Failed to update application property. Error detail: Conflicting property values. 'SignInAudience' must be 'PersonalMicrosoftAccount' for 'windows' property to be set."

I will try to get a test tenant with a "verifiedCustomDomain" to be able to test the remaining formats.

jurgenrapp commented 8 months ago

@bgavrilMS Hi, we have now tested the remaining formats with the same result. https://<verifiedCustomDomain>/<string> https://<string>.<verifiedCustomDomain> https://<string>.<verifiedCustomDomain>/<string>

So the issue remains we can't get Expose an API scopes to work with .WithBroker.

bgavrilMS commented 8 months ago

Ok, I will bump this to a P1 as it blocks a major scenario - client calls "your own web api, protected by Azure AD".

bgavrilMS commented 8 months ago

@jurgenrapp - I just tried this with a brand new client and mid-tier app and it seems to have worked fine.

And when I requested a token with WAM for a work and school account I got:

Can you please enable PII logging so that we can see the error message associated with the exception? https://learn.microsoft.com/en-us/entra/msal/dotnet/advanced/exceptions/msal-logging?tabs=dotnet
Notice that this is a v2 token because new apps are configured as v2. If yours is v1, consider updating the manifest to make it v2.

jurgenrapp commented 8 months ago

@bgavrilMS

First I changed accessTokenAcceptedVersion to 2, signInAudience is set to "AzureADMultipleOrgs" don't know if that changes anything.

Then I enabled logging and got some clues. I got "AADSTS90009: Application 'd220846b-1916-48d2-888b-9e16f6d9848b' is requesting a token for itself. This scenario is supported only if resource is specified using the GUID based App Identifier."

Somewhere I found that this was caused by requesting a scope starting with api://

So I changed the requested scope to just d220846b-1916-48d2-888b-9e16f6d9848b/x for some reason that forced med to re consent the application. Event though I had just consented it.

After the reconsent I now got both the custom api scope combined with the applications graph scopes in the authresult which I have never seen before. And was quite excited about because that would be really neat. A couple of hours later it once again wants me to reconsent the app when I request d220846b-1916-48d2-888b-9e16f6d9848b/x.

However if I now try to acquire a token for api;//d220846b-1916-48d2-888b-9e16f6d9848b/x it works the old way, I only get the requested scope in the scopes list and without reconsent.

So my original 4 test are now working, so I guess it works. But I am a bit confused, will do some more testing to see if I can make any sense of it. I guess the Issue can be closed.

I'm very grateful for you help.

Kind regards

bgavrilMS commented 8 months ago

Ok, I'm glad you have a path forward at least. It's still not great that the api:// is not working, but I think separating your client from your web api would also work? And it's not bad to have them separate.

In terms of scopes, note that AAD will issue a token for all the scopes the user has consented to. For example, if you consent to "User.Read", "Files.Read", "Files.Write" and you request a token just for "User.Read", then you'll get a token for all "User.Read", "Files.Read", "Files.Write". This is an optimization that AAD made some time back to minimize the number of roundtrips.

In terms of consent, I am surprised it asked you to reconsent. Let me know if you find anything else strange.

jurgenrapp commented 8 months ago

@bgavrilMS

Regarding you question: We have two applications one client application and one web-api-application the client-application has a permission link to the web-apllication-app, Not visible in this test app manifest though because outside of the scope for this test.

The only reason we expose an api in the client-app was because we couldn't get MSAL to work without it, when upgrading from ADAL and Internet told us to solve it that way.

My investigation: I created a new app and got the same, to me, strange behavior. Is this how it is supposed to work?

See the full app-manifest at the bottom of this post.

The app is consented by the customer via a consent link similar to https://login.microsoftonline.com/common/adminconsent?client_id=xxx.
I tried to AcquireTokenInteractive with scope "api://{clientId}/test" this fails like last time with "AADSTS90009: Application '...' is requesting a token for itself. This scenario is supported only if resource is specified using the GUID based App Identifier.
I change AcquireTokenInteractive so that it uses scope "{clientId}/test" no api-prefix, this triggers consent popup after the app has been consented via the popup I get these permissions in the customers Enterprise application.:

This is a big issue since this isn't the same permissions we get if we use the consent link and is not a viable solution for our customers
AcquireTokenInteractive with scope "{clientId}/test" returns:

Looks kinda strange because the User.Read scope is a Microsoft scope but is now added to the test application.

I now request the original scope with the api-prefix "api://{clientId}/test" and now that also works and I get the expected scope back:
Everything seems to be working now but as a final test I reconsent the enterprise app in the customers aad: Now all the extra permissions that was added by the "live" consent popup are gone:
Trying to once again request the original scope that gave us AADSTS90009 in the beginning with the api-prefix for whatever reason is now working as expected:
So somehow the popup reconsent changes something somewhere so that the app works as expected. But I don't think we can do that extra step with our customers.


    "id": "9e2458a2-ebe7-4002-83d1-47a5d73daa5b",
    "acceptMappedClaims": null,
    "accessTokenAcceptedVersion": 2,
    "addIns": [],
    "allowPublicClient": null,
    "appId": "b3cb02b9-75a0-4c9d-a23d-7c7e3ae8976c",
    "appRoles": [],
    "oauth2AllowUrlPathMatching": false,
    "createdDateTime": "2023-12-18T20:06:05Z",
    "description": null,
    "certification": null,
    "disabledByMicrosoftStatus": null,
    "groupMembershipClaims": null,
    "identifierUris": [
        "api://b3cb02b9-75a0-4c9d-a23d-7c7e3ae8976c"
    ],
    "informationalUrls": {
        "termsOfService": null,
        "support": null,
        "privacy": null,
        "marketing": null
    },
    "keyCredentials": [],
    "knownClientApplications": [],
    "logoUrl": null,
    "logoutUrl": null,
    "name": "mytest testapp",
    "notes": null,
    "oauth2AllowIdTokenImplicitFlow": false,
    "oauth2AllowImplicitFlow": false,
    "oauth2Permissions": [
        {
            "adminConsentDescription": "test",
            "adminConsentDisplayName": "test",
            "id": "779517a6-9235-4624-b276-40147e611e7b",
            "isEnabled": true,
            "lang": null,
            "origin": "Application",
            "type": "Admin",
            "userConsentDescription": null,
            "userConsentDisplayName": null,
            "value": "test"
        }
    ],
    "oauth2RequirePostResponse": false,
    "optionalClaims": null,
    "orgRestrictions": [],
    "parentalControlSettings": {
        "countriesBlockedForMinors": [],
        "legalAgeGroupRule": "Allow"
    },
    "passwordCredentials": [],
    "preAuthorizedApplications": [],
    "publisherDomain": "mytest.onmicrosoft.com",
    "replyUrlsWithType": [
        {
            "url": "https://login.microsoftonline.com/common/oauth2/nativeclient",
            "type": "InstalledClient"
        },
        {
            "url": "ms-appx-web://microsoft.aad.brokerplugin/b3cb02b9-75a0-4c9d-a23d-7c7e3ae8976c",
            "type": "InstalledClient"
        }
    ],
    "requiredResourceAccess": [
        {
            "resourceAppId": "00000003-0000-0000-c000-000000000000",
            "resourceAccess": [
                {
                    "id": "e1fe6dd8-ba31-4d61-89e7-88639da4683d",
                    "type": "Scope"
                }
            ]
        }
    ],
    "samlMetadataUrl": null,
    "signInUrl": null,
    "signInAudience": "AzureADMultipleOrgs",
    "tags": [
        "apiConsumer",
        "desktopApp"
    ],
    "tokenEncryptionKeyId": null
}```

jurgenrapp commented 8 months ago

@bgavrilMS

Also we tried doing it the correct way and requested the web-api scope exposed in the linked web-api-app instead of requesting a exposed scope from the client-app and then it worked at once. So doing it the correct way seems to be a better approach.

Once again thank you for your help.

Kind regards

bgavrilMS commented 8 months ago

@bgavrilMS

Also we tried doing it the correct way and requested the web-api scope exposed in the linked web-api-app instead of requesting a exposed scope from the client-app and then it worked at once. So doing it the correct way seems to be a better approach.

Once again thank you for your help.

Kind regards

Thanks for providing this. What is the app registration change you had to make to get this going?

jurgenrapp commented 8 months ago

@bgavrilMS We have two applications the client-app and the api/web-app.

The client-app has the user_impersonation scope linked/added from the web-app

The webapp:

In our original setup, the client-app also exposed an api and when we signed in we requested a scope from the client-app. So IPublicClientApplication was created with the same appid as we requested the scope from via Expose an API. This setup we never really got to work with Broker but it was working before the switch to broker.

The changes we made to make it work:

Now when we sign in the IPublicClientApplication is pointing to the client-app but the scope requested with AcquireTokenInteractive is located in the web/api-app. This was the silver bullet.

Changes that it is unclear if they made any difference:

We removed the Expose an API from the Client-app.
We changed the accessTokenAcceptedVersion to 2.
Since we are using WithDefaultRedirectUri we had to add the ms-appx-web: redirect uri to the client-app.

I hope this answers your question, if not feel free to ask me more specific questions. Kind regards

AzureAD / microsoft-authentication-library-for-dotnet