search

microsoft / Partner-Center-PowerShell

PowerShell module for managing Partner Center resources.
https://docs.microsoft.com/powershell/partnercenter/
MIT License
130 stars 59 forks source link

New/unknown error - New-PartnerAccessToken : AADSTS530034: A delegated administrator was blocked from accessing the tenant due to account risk. #389

Closed sansbacher closed 2 years ago

sansbacher commented 2 years ago

Hi!

I think I have found a NEW/UNKNOWN error. It is NOT documented in Microsoft's list of AAD errors. And I can find NO search hits for it using Google, Bing, Issues in this repo, or on the MS Partner Community forums for the Secure App Model. However I did just make a new post about this there too. The error is:

New-PartnerAccessToken : AADSTS530034: A delegated administrator was blocked from accessing the tenant due to account risk.

This error started for me on Feb 3, 2022. It happens when I try to get a new Access Token in order to connect to our Customers' Tenants (for which we have Delegated Admin Privileges via the Partner Center). It started happening to 2 of our customers, then 20, then 40, now over 50! It does not affect all Customers (some I can connect to just fine), some customers fail for any AccessToken, some work for AAD or Exch Online or MS Azure.

We've made no recent changes (the most recent was just over 3 months ago (I changed from using my account as the PartnerCenter AdminAgent account/UPN to a common MFA-protected admin account). Most customers don't have AAD P2 licenses (neither do we) so no Risk-Based login blocking. I can renew my Refresh Tokens just fine. I've confirmed that when it works the AccessToken's AMR contains 'mfa'. I have tried removing and re-adding Delegated Admin Privileges, no help.

Steps to reproduce

Connecting to MY (CSP) tenancy to get a list of all my Customers works:

# This WORKS fine:
$aadGraphToken = New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $RefreshToken -Scopes 'https://graph.windows.net/.default' -ServicePrincipal -Tenant $AzureADTenantId
$graphToken = New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $RefreshToken -Scopes 'https://graph.microsoft.com/.default' -ServicePrincipal -Tenant $AzureADTenantId
Connect-AzureAD -AadAccessToken $aadGraphToken.AccessToken -AccountId $PartnerCenterMfaUpn -MsAccessToken $graphToken.AccessToken -TenantId $AzureADTenantId
$AllCustomers = Get-AzureADContract -All $True     # Gets a list of all our Customers

$AzureADAppId is our Azure AD Partner Center Native App ID, $pcCreds holds the AAD AppId and the Secret (which is not expired), $RefreshToken is the long refresh token (which we renew every week), $AzureADTenantId is OUR (CSP) Tenant ID, $PartnerCenterMfaUpn is the account with Admin Agent role (ie o365admin@ourDomain.com) that has MFA.

However connecting to my CUSTOMERS' tenancies will fail:

Any of these will produce the AADSTS530034 error for SOME customers (not all)

$theCustomer = $AllCustomers[0]   # ASSUMING customer 0 is one of the ones experiencing the issue

# These can all produce the AADSTS530034 error, though not all customers error on every one:
$aadTok = New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $refreshToken -Scopes "https://graph.windows.net/.default" -ServicePrincipal -Tenant $theCustomer.CustomerContextId
# or
$graphTok = New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $refreshToken -Scopes "https://graph.microsoft.com/.default" -ServicePrincipal -Tenant $theCustomer.CustomerContextId
# or
$azureToken = New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $refreshToken -Scopes 'https://management.azure.com/user_impersonation' -ServicePrincipal -Tenant $theCustomer.CustomerContextId
# or
$exTok = New-PartnerAccessToken -Tenant $theCustomer.CustomerContextId -RefreshToken $ExchangeRefreshToken -Scopes "https://outlook.office365.com/.default" -ApplicationId "a0c73c16-a7e3-4564-9a95-2bdf47383716"

$theCustomer.CustomerContextId has the CUSTOMER'S Tenant ID, other variables are as above.

The above New-PartnerAccessToken commands DO work for some of our Customers just fine.

Expected behavior

The New-PartnerAccessToken command should return an Object with several Properties, such as RefreshToken and most importantly AccessToken - provided we have Delegated Admin Privileges to their Tenancy, which I have confirmed we do (ie. I can manage their Azure, AAD, Exch Online, etc using the Partner Center portal and I've confirmed we are listed as having Global Admin role in their tenancy).

Also, any error messages returned should be listed in official documentation online.

Actual behavior

For some, not all, customers he command is returning a new error that does not seem to exist - at least that I can find. An example of the error is:

New-PartnerAccessToken : AADSTS530034: A delegated administrator was blocked from accessing the tenant due to account risk.
Trace ID: 60b5dbaa-5b7b-4a3b-aa37-6eece32d3f01
Correlation ID: 79dd5a9d-3df0-4a80-a709-4353af7c0e12
Timestamp: 2022-02-08 16:39:35Z
At line:1 char:11
+ $aadTok = New-PartnerAccessToken -ApplicationId $AzureADAppId -Creden ...
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : CloseError: (:) [New-PartnerAccessToken], MsalServiceException
    + FullyQualifiedErrorId : Microsoft.Store.PartnerCenter.PowerShell.Commands.NewPartnerAccessToken

Diagnostic logs

I have tried running the New-PartnerAccessToken with the -debug switch, I removed the confirmation prompts, here's the log:

New-PartnerAccessToken -ApplicationId $AzureADAppId -Credential $pcCreds -RefreshToken $refreshToken -Scopes "https://graph.windows.net/.default" -ServicePrincipal -Tenant $Customer.CustomerContextId -Environment AzureCloud -Debug

DEBUG: [RefreshTokenAuthenticator] Calling GetAccountsAysnc
DEBUG: [RefreshTokenAuthenticator] Calling AcquireTokenByRefreshToken - Scopes: 'https://graph.windows.net/.default'
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:09 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) MSAL MSAL.CoreCLR with assembly version '4.8.1.0'.
CorrelationId(fe9ec0e9-c26f-4879-89dc-1c2fe2788986)
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:09 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) Using 1 scopes for acquire token by refresh token request
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:09 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0)
=== Request Data ===
Authority Provided? - True
Scopes - https://graph.windows.net/.default
Extra Query Params Keys (space separated) -
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:09 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) === Token Acquisition (ByRefreshTokenRequest) started:

 Authority Host: login.microsoftonline.com
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:09 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) Resolving authority endpoints... Already resolved? - TRUE
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:10 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) Response status code does not indicate success: 400 (BadRequest).
DEBUG: [MSAL] Info (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:10 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) HttpStatusCode: 400: BadRequest
DEBUG: [MSAL] Error (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:10 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) Exception type: Microsoft.Identity.Client.MsalServiceException
, ErrorCode: invalid_request
HTTP StatusCode 400
CorrelationId fe9ec0e9-c26f-4879-89dc-1c2fe2788986
DEBUG: [MSAL] Error (False) MSAL 4.8.1.0 MSAL.CoreCLR N/A [02/08/2022 16:47:10 - fe9ec0e9-c26f-4879-89dc-1c2fe2788986]
(UnknownClient: 0.0.0.0) Exception type: Microsoft.Identity.Client.MsalServiceException
, ErrorCode: invalid_request
HTTP StatusCode 400
CorrelationId fe9ec0e9-c26f-4879-89dc-1c2fe2788986

   at Microsoft.Identity.Client.OAuth2.OAuth2Client.ThrowServerException(HttpResponse response, RequestContext
requestContext)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.CreateResponse[T](HttpResponse response, RequestContext
requestContext, Boolean addCorrelationId)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.<ExecuteRequestAsync>d__10`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.OAuth2.OAuth2Client.<GetTokenAsync>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<SendHttpMessageAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<SendTokenRequestAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.ByRefreshTokenRequest.<ExecuteAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Identity.Client.Internal.Requests.RequestBase.<RunAsync>d__14.MoveNext()

AADSTS530034: A delegated administrator was blocked from accessing the tenant due to account risk.
Trace ID: e538b881-9a1c-4277-85dc-e8f0977c3e01
Correlation ID: fe9ec0e9-c26f-4879-89dc-1c2fe2788986
Timestamp: 2022-02-08 16:47:10Z

Environment

I am using PartnerCenter PowerShell Module 3.0.10.
It happens on my machine (Win 10 21H1 x64) and Windows PowerShell 5.1. It also happens on our automation server running Server 2012 R2 and Win PowerShell 5.1. This has happened at different times, on different days since Feb 3.

If someone can suggest what I am doing wrong, what may have changed (I have looked but not found anything), or what I need to change I am all ears! I'm also happy to try/test things if you think it will help. If I've missed anything in reporting this please let me know.

Thanks so much! --Saul

sansbacher commented 2 years ago

This can be closed - I found the solution. I'll leave this here since the AADSTS530034 error doesn't appear anywhere else and maybe someone else will encounter it and stumble upon this Issue.

Janosch Ulmer (MS) in the MS Partner Community forum helped point me in the right direction. He suggested looking at Risk Detections (in case Security Defaults was enabled). None of the customers I checked had Security Defaults enabled and neither did we (the MSP/CSP) - as we have Conditional Access policies. And none of us had any Risk Detections related to the account.

BUT... under "Risky users" I did find the Admin account I was using (the UPN used in $PartnerCenterMfaUpn) listed - but Risk last updated was Feb 2020 - which was ages ago! I have no idea why it suddenly became an issue.

Solution (at least for me)

Go to Azure AD in your Tenant (our CSP/MSP Tenant - the one $AzureADTenantId points to), Azure Active Directory, Security, Risky users. Find the account, check the box and click 'Dismiss user(s) risk' and click 'Yes'. Wait a few mins and then try connecting to your Customers using your script.

It worked for me. And I swear I had looked there earlier...