search

Azure / azure-cli

Azure Command-Line Interface
MIT License
3.98k stars 2.95k forks source link

Azure CLI task fails with `AADSTS700024` after 5 minutes #28737

Closed jiasli closed 5 months ago

jiasli commented 5 months ago

MSAL introduced a regression in 1.27.0 (https://github.com/AzureAD/microsoft-authentication-extensions-for-python/issues/127, https://github.com/AzureAD/microsoft-authentication-library-for-python/pull/644) which is adopted by Azure CLI 2.59.0 (https://github.com/Azure/azure-cli/pull/28556).

This regression makes MSAL's ConfidentialClientApplication bypass msal_extensions.token_cache.PersistedTokenCache, so access tokens are no longer retrieved from the token cache. Instead, every command now retrieves a new access token from the AAD Security Token Service (STS). Any commands fails with AADSTS700024 after the ID token expires (5 minutes on GitHub Actions, 10 minutes on Azure DevOps).

Originally posted by @jiasli in https://github.com/Azure/azure-cli/issues/28708#issuecomment-2049400226

jiasli commented 5 months ago

Workarounds

Adopt either workarounds:

  1. Use service principal secret for authentication:

  2. [GitHub Actions only] In Azure CLI Action azure/cli@v2, specify azcliversion to use an older version of Azure CLI below 2.59.0, such as 2.58.0: https://github.com/marketplace/actions/azure-cli-action

jiasli commented 5 months ago

We plan to fix this issue in the next Azure CLI release: https://github.com/Azure/azure-cli/milestone/141

Official Release: 04/30/2024 Azure CLI version: 2.60.0

For now, to get unblocked, please follow the instructions at https://github.com/Azure/azure-cli/issues/28737#issuecomment-2051026300.

andre-qumulo commented 5 months ago

@jiasli Thank you for the update. Just for clarity, what, specifically, will get fixed? Will we still need the code to continue requesting the OID token in the background or will we just need to use Azure CLI 2.60.0?

danwkennedy commented 5 months ago

Is it possible to release this as a patch fix sooner? This potentially impacts every use of the azure/login GitHub Action.

Also, the workaround only works if you're using the azure/cli Action. If you use the az command directly in your scripts, it won't work. This use case is fairly common since hosted GitHub Actions runners come with the latest version of the az command preinstalled. Without this, users have to override the version and we have to hope they remember to undo their workarounds later.

danwkennedy commented 5 months ago

@jiasli would it be possible to release this in a patch fix?

cloudcosmonaut commented 5 months ago

@jiasli is there anyway to promote this as a hotfix so releases that rely on az cli can work again? The current time limit is braking a lot of builds :(

yonzhan commented 5 months ago

@jiasli is working on the new build right now.

cloudcosmonaut commented 5 months ago

Awesome! Any ETA for this release?

vaibbavisk20 commented 5 months ago

Is there any workaround for Azure/login@v2.1.0 while the hot fix makes it to production, since there doesn't seem to be a way to change the cli version this action uses?

vaibbavisk20 commented 5 months ago

Service principal with a secret is not feasible for our case, due to issues transmitting and storing the value. Changing azure/cli version is not feasible due to using azure/login and azure/powershell only

jiasli commented 5 months ago

Changing azure/cli version is not feasible due to using azure/login and azure/powershell only

This 5-minute-expiration issue only affects azure/cli action. azure/login and azure/powershell are not affected. If your task lasts longer than 60 minutes, this is currently a known limitation: https://github.com/Azure/azure-cli/issues/28708

cloudcosmonaut commented 5 months ago

@jiasli is working on the new build right now.

@yonzhan any update on this? When can we expect a release?

yonzhan commented 5 months ago

Build to Cloud Shell: 04/25/2024 Official Release: 04/30/2024 Azure CLI version: 2.60.0

nosalan commented 5 months ago

This is quite a problematic issue for us and makes service connections based on Workload Identity federation unusable. Please add a test suite for regression so that it doesn't happen again.

nosalan commented 5 months ago

The issue also happens with Azure PowerShell on the latest MS Hosted Azure DevOps agents, example:

Set-AzFirewall -AzureFirewall $azfwAllocated | Select-Object Name,  …
     |    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | ClientAssertionCredential authentication failed: A configuration issue
     | is preventing authentication - check the error message from the server
     | for details. You can modify the configuration in the application
     | registration portal. See https://aka.ms/msal-net-invalid-client for
     | details.  Original exception: AADSTS700024: Client assertion is not
     | within its valid time range. Current time: 2024-04-26T07:47:14.2804194Z,
     | assertion valid from 2024-04-26T06:52:04.0000000Z, expiry time of
     | assertion 2024-04-26T07:02:04.0000000Z. Review the documentation at
     | https://docs.microsoft.com/azure/active-directory/develop/active-directory-certificate-credentials . Trace ID: <REDACTED> Correlation ID: <REDACTED>

Also AzureCli@2 task is affected.

cloudcosmonaut commented 5 months ago

Build to Cloud Shell: 04/25/2024 Official Release: 04/30/2024 Azure CLI version: 2.60.0

Can I fetch the Cloud build to ingest it in a pipeline decorator?

jiasli commented 4 months ago

Azure CLI 2.60.0 has been released just now with this issue fixed:

https://github.com/Azure/azure-cli/blob/dev/src/azure-cli-core/HISTORY.rst#2600

Fix #28737: Fix token cache for service principal authentication (#28747)

The rollout status for Azure CLI on GitHub Actions and Azure DevOps images can be found at https://github.com/actions/runner-images

bcarthic commented 4 months ago

@jiasli, Even after upgrading to Azure CLI 2.60.0, I am facing the same issue. I am running the Azure CLI task from Azure devops and it expires after 10 mins and I get ERROR: AADSTS700024: Client assertion is not within its valid time range.

Since the images still uses 2.59.0, I do az upgrade --yes before running dotnet test.

LiliDeng commented 4 months ago

@jiasli, Even after upgrading to Azure CLI 2.60.0, I am facing the same issue. I am running the Azure CLI task from Azure devops and it expires after 10 mins and I get ERROR: AADSTS700024: Client assertion is not within its valid time range.

Since the images still uses 2.59.0, I do az upgrade --yes before running dotnet test.

+1

jiasli commented 4 months ago

@bcarthic, are you requesting a data-plane access token? If so, please see https://github.com/Azure/azure-cli/issues/28708#issuecomment-2047256166.

m-soltani commented 4 months ago

@bcarthic, are you requesting a data-plane access token? If so, please see #28708 (comment).

@jiasli What happens if I intend to kick off a long-running operation that starts by obtaining an access token and concludes five hours later? This scenario is typical for us, as we use Packer to create VM images for our build servers. Are there any plans to implement a process in Azure CLI that automatically exchanges a refresh token for an access token in the background? Packer is configured to use already logged in account via Azure CLi. With the current fix, the access token expires 1 Hour later and there is no chance to refreshing that :(

jiasli commented 4 months ago

Azure CLI 2.60.0 has been deployed to GitHub Actions and Azure DevOps images: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md#cli-tools