search

microsoft / vscode-vsce

VS Code Extension Manager
https://code.visualstudio.com/
Other
819 stars 208 forks source link

"The requested operation is not allowed" when using --azure-credential #976

Closed winstliu closed 4 months ago

winstliu commented 6 months ago

Super excited to see vsce support Entra auth!

I was trying it today but ran into the following error:

 INFO  Publishing 'redacted.redacted v1.2024.510002'...
 ERROR  {"$id":"1","innerException":null,"message":"The requested operation is not allowed.","typeName":"Microsoft.TeamFoundation.Framework.Server.InvalidAccessException, Microsoft.TeamFoundation.Framework.Server","typeKey":"InvalidAccessException","errorCode":0,"eventId":3000}

The publish command is vsce publish --azure-credential --packagePath $(Pipeline.Workspace)\drop_build_main\redacted.vsix

The service principal does have the right permissions on the marketplace publisher.

lszomoru commented 6 months ago

The service principal does have the right permissions on the marketplace publisher.

If this is correct, why do you think that this is an issue related to vsce and not a permission issue that is outside the scope of vsce? Thanks!

winstliu commented 6 months ago

1) The error message is quite generic, was output by vsce, and doesn't tell me that it's not due to a problem with vsce. 2) When publishing VS extensions via VsixPublisher, I am able to publish successfully with the same account to the same publisher.

Thus as other publishing tools succeed using the same service principal, but vsce doesn't, I suspect vsce.

(edit: to be clear just in case there was a misunderstanding, the service principal has Owner permissions and should have full privileges to the publisher)

lszomoru commented 6 months ago

Based on the exception type, it seems to be coming from the Marketplace (Microsoft.TeamFoundation.Framework.Server.InvalidAccessException). How are you trying to publish your VS Code extension? What Azure Pipeline task are you using to run vsce?

winstliu commented 6 months ago

AzureCLI. The entirety of the task is as follows:

  - task: AzureCLI@2
    inputs:
      azureSubscription: Azure Marketplace Service Connection
      scriptType: pscore
      scriptLocation: inlineScript
      inlineScript: >
        vsce publish
        --azure-credential
        --packagePath $(Pipeline.Workspace)\drop_build_main\redacted.vsix
        $(extraVsceArgs)
    displayName: Publish to Marketplace

Azure Marketplace Service Connection is connected to the service principal on the Marketplace account (this same service connection is also used for our VS publishing flows and those work, so it's not a problem with the connection), and extraVsceArgs is either --pre-release or nothing at all.

lszomoru commented 6 months ago

Could you try adding the addSpnToEnvironment: true to the task?

winstliu commented 5 months ago

Exact same failure, unfortunately.

naiyunzhang commented 5 months ago

I encountered the exact same failure. Is there any update on this issue?

I truly appreciate the time and effort you dedicate to maintaining this project.

dtivel commented 5 months ago

vsce's --azure-credential option uses Azure SDK for JS's DefaultAzureCredential to authenticate.

DefaultAzureCredential will attempt to authenticate using a fixed order of authentication methods. It will stop on the first one that succeeds in obtaining an access token.

It may be that the authentication method that is succeeding in obtaining an access token is not the authentication method you expect, and that access token doesn't have the publish privileges you need.

You could try setting AZURE_LOG_LEVEL to info (per these instructions). The verbose logging would tell you how DefaultAzureCredential is authenticating. I haven't tried this myself, but it looks like it should work. It works.

Also, you could explicitly exclude unwanted authentication methods before your desired authentication method. For example, if you're using Azure CLI to login before calling vsce publish you could try excluding other authentication methods before it. (The ExcludeXXXCredential property names listed here can be set as environment variables.)

Setting these environment variables does not work.

$Env:ExcludeEnvironmentCredential='true'
$Env:ExcludeWorkloadIdentityCredential='true'
$Env:ExcludeManagedIdentityCredential='true'
$Env:ExcludeSharedTokenCacheCredential='true'
$Env:ExcludeVisualStudioCredential='true'
$Env:ExcludeVisualStudioCodeCredential='true'
trevors20 commented 5 months ago

I too am having the exact same issue when I've tried to use the --azure-credential switch. As @dtivel describes, I suspect that some weird and unknown identity is being found and being used which does not have access to the marketplace.
I'm going to try these environment variables and give it a try. Thanks @dtivel for this important information. It helps a lot.

CC: @benibenj , @lszomoru consider documenting this in the vsce documentation?

dtivel commented 5 months ago

@trevors20, thanks much and please let us know if this works. I'm just guessing at this point, so having confirmation on a solution would help a lot. I agree, if this is the problem, we should update documentation with information about this.

dtivel commented 5 months ago

I updated my comment above. My suggested workaround does not work. I believe the root cause is still valid.

I tried the ExcludeXXXCredential environment variables, and I still see in verbose logs DefaultAzureCredential attempting authentication methods I excluded. I also hacked vsce locally to exclude unwanted authentication options in code. That also didn't work as expected.

It would appear that, contrary to documentation, the Azure SDK for JS does not support the ExcludeXXXCredential options., so I opened https://github.com/Azure/azure-sdk-for-js/issues/30119.

Even if I'm mistaken about ExcludeXXXCredential support, it looks like the SDK doesn't honor the options when set through environment variables --- only through code. (My first-hand experience with the Azure SDK for .NET is that did at least at one time.) That means a code change would be required to vsce to propagate ExcludeXXXCredential options, assuming they actually work.

dtivel commented 5 months ago

It has been confirmed: Azure SDK for JS did not actually implement ExcludeXXXCredential support.

lszomoru commented 5 months ago

@dtivel, thank you very much for looking into this. When you enabled info logging using AZURE_LOG_LEVEL what is the authentication method that is being used? Would you be able to share those logs privately? Thanks! I do agree with you that a ChainedTokenCredential is the right solution but I would like to see if there is a way to set the correct order that would work out of the box without additional configuration.

Until we get to the bottom of this you can workaround by getting a token using az login and your choice of auth, and then use that token with the --pat option of vsce. Please give it a try and let me know how it goes.

Couple of troubleshooting steps that can come in handy:

  1. After getting the token dump the Azure DevOps profile associated with the token to ensure that it is the expected user: 
    az rest -u https://app.vssps.visualstudio.com/_apis/profile/profiles/me --resource 499b84ac-1321-427f-aa17-267ca6975798
  2. After getting the token verify the Marketplace permissions of the token: 
    vscode verify-pat --pat <token>
SBalslev commented 5 months ago

Have the same issue and can confirm that getting a token and using with --pat works as a workarround.

does not work vsce publish --pre-release --azure-credential --packagePath $alVsix[0].FullName works $aadToken = az account get-access-token --query accessToken --resource 499b84ac-1321-427f-aa17-267ca6975798 -o tsv vsce publish --pre-release --packagePath $alVsix[0].FullName --pat $aadToken

Also tried same workarounds as above without any resolution for --azrure-credential. $env:AZURE_LOG_LEVEL = "info" $env:ExcludeEnvironmentCredential='true' $env:ExcludeWorkloadIdentityCredential='true' $env:ExcludeManagedIdentityCredential='true' $env:ExcludeSharedTokenCacheCredential='true' $env:ExcludeVisualStudioCredential='true' $env:ExcludeVisualStudioCodeCredential='true'

lszomoru commented 5 months ago

@SBalslev thanks for trying out the workaround and for confirming that it is working. Would you be able to share the logs that are being generated when you enable the AZURE_LOG_LEVEL? I would like to understand which identity is being picked to be able to implement the correct long-term solution. Thanks!

SBalslev commented 5 months ago

Here's the log from the last run before I switched to PAT. This run has all the variables set to true.

2024-06-19T21:41:22.6286496Z ##[section]Starting: Publish with managed identity
2024-06-19T21:41:22.6294661Z ==============================================================================
2024-06-19T21:41:22.6294868Z Task         : Azure CLI
2024-06-19T21:41:22.6294967Z Description  : Run Azure CLI commands against an Azure subscription in a PowerShell Core/Shell script when running on Linux agent or PowerShell/PowerShell Core/Batch script when running on Windows agent.
2024-06-19T21:41:22.6295325Z Version      : 2.241.0
2024-06-19T21:41:22.6295447Z Author       : Microsoft Corporation
2024-06-19T21:41:22.6295554Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
2024-06-19T21:41:22.6295728Z ==============================================================================
2024-06-19T21:41:23.8305333Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" --version"
2024-06-19T21:41:25.4998295Z azure-cli                         2.61.0
2024-06-19T21:41:25.5019504Z 
2024-06-19T21:41:25.5133373Z core                              2.61.0
2024-06-19T21:41:25.5167696Z telemetry                          1.1.0
2024-06-19T21:41:25.7172023Z 
2024-06-19T21:41:25.8244256Z Dependencies:
2024-06-19T21:41:25.8312684Z msal                              1.28.0
2024-06-19T21:41:25.8317606Z azure-mgmt-resource               23.1.1
2024-06-19T21:41:25.8677162Z 
2024-06-19T21:41:25.8719520Z Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
2024-06-19T21:41:25.8787456Z Extensions directory 'C:\Users\cloudtest\.azure\cliextensions'
2024-06-19T21:41:25.8792526Z 
2024-06-19T21:41:25.9102932Z Python (Windows) 3.11.8 (tags/v3.11.8:db85d51, Feb  6 2024, 21:52:07) [MSC v.1937 32 bit (Intel)]
2024-06-19T21:41:25.9106113Z 
2024-06-19T21:41:25.9108205Z Legal docs and information: aka.ms/AzureCliLegal
2024-06-19T21:41:25.9115340Z 
2024-06-19T21:41:25.9123041Z 
2024-06-19T21:41:25.9123760Z Your CLI is up-to-date.
2024-06-19T21:41:25.9126075Z Setting AZURE_CONFIG_DIR env variable to: D:\a\_work\_temp\.azclitask
2024-06-19T21:41:25.9127373Z Setting active cloud to: AzureCloud
2024-06-19T21:41:25.9135026Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" cloud set -n AzureCloud"
2024-06-19T21:41:26.6405564Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" login --service-principal -u *** --tenant 72f988bf-86f1-41af-91ab-2d7cd011db47 --allow-no-subscriptions --federated-token ***"
2024-06-19T21:41:28.2693156Z [
2024-06-19T21:41:28.2713794Z   {
2024-06-19T21:41:28.2824710Z     "cloudName": "AzureCloud",
2024-06-19T21:41:28.2860785Z     "homeTenantId": "72f988bf-86f1-41af-91ab-2d7cd011db47",
2024-06-19T21:41:28.4869395Z     "id": "35afe423-9efe-4d80-a8b7-7a5281e7fedf",
2024-06-19T21:41:28.4956048Z     "isDefault": true,
2024-06-19T21:41:28.5023476Z     "managedByTenants": [],
2024-06-19T21:41:28.5028793Z     "name": "BC_DEV_DeveloperTools",
2024-06-19T21:41:28.5815317Z     "state": "Enabled",
2024-06-19T21:41:28.5820306Z     "tenantId": "72f988bf-86f1-41af-91ab-2d7cd011db47",
2024-06-19T21:41:28.5821497Z     "user": {
2024-06-19T21:41:28.5823377Z       "name": "***",
2024-06-19T21:41:28.5823866Z       "type": "servicePrincipal"
2024-06-19T21:41:28.5824278Z     }
2024-06-19T21:41:28.5824663Z   }
2024-06-19T21:41:28.5825043Z ]
2024-06-19T21:41:28.5859613Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account set --subscription 35afe423-9efe-4d80-a8b7-7a5281e7fedf"
2024-06-19T21:41:29.3667237Z [command]"C:\Program Files\PowerShell\7\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_work\_temp\azureclitaskscript1718833283196.ps1'"
2024-06-19T21:41:30.2195286Z Publishing D:\a\_work\r1\a\_Microsoft_Dynamics_Nav_VsCodeAlExtension\content\al-14.0.1053825-marketplace.vsix
2024-06-19T21:41:31.2984466Z azure:identity:info EnvironmentCredential => Found the following environment variables: 
2024-06-19T21:41:31.2987337Z azure:identity:info WorkloadIdentityCredential => Found the following environment variables: 
2024-06-19T21:41:31.2992536Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-06-19T21:41:31.3006095Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-06-19T21:41:31.3028105Z azure:identity:info WorkloadIdentityCredential => WorkloadIdentityCredential: is unavailable. tenantId, clientId, and federatedTokenFilePath are required parameters. 
2024-06-19T21:41:31.3028948Z       In DefaultAzureCredential and ManagedIdentityCredential, these can be provided as environment variables - 
2024-06-19T21:41:31.3029457Z       "AZURE_TENANT_ID",
2024-06-19T21:41:31.3029816Z       "AZURE_CLIENT_ID",
2024-06-19T21:41:31.3030393Z       "AZURE_FEDERATED_TOKEN_FILE". See the troubleshooting guide for more information: https://aka.ms/azsdk/js/identity/workloadidentitycredential/troubleshoot  
2024-06-19T21:41:31.3041688Z azure:identity:info ManagedIdentityCredential - Azure Arc MSI => ManagedIdentityCredential - Azure Arc MSI: The environment variables needed are: IMDS_ENDPOINT and IDENTITY_ENDPOINT
2024-06-19T21:41:31.3049081Z azure:identity:info ManagedIdentityCredential - Fabric MSI => ManagedIdentityCredential - Fabric MSI: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT, IDENTITY_HEADER and IDENTITY_SERVER_THUMBPRINT
2024-06-19T21:41:31.3050916Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2019 => ManagedIdentityCredential - AppServiceMSI 2019: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT and IDENTITY_HEADER.
2024-06-19T21:41:31.3052058Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2017 => ManagedIdentityCredential - AppServiceMSI 2017: Unavailable. The environment variables needed are: MSI_ENDPOINT and MSI_SECRET.
2024-06-19T21:41:31.3053225Z azure:identity:info ManagedIdentityCredential - CloudShellMSI => ManagedIdentityCredential - CloudShellMSI: Unavailable. The environment variable MSI_ENDPOINT is needed.
2024-06-19T21:41:31.3054546Z azure:identity:info ManagedIdentityCredential - Token Exchange => ManagedIdentityCredential - Token Exchange: Unavailable. The environment variables needed are: AZURE_CLIENT_ID (or the client ID sent through the parameters), AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE
2024-06-19T21:41:31.3063767Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Pinging the Azure IMDS endpoint
2024-06-19T21:41:31.3078463Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request 1f5ca017-909b-4b92-bcb2-0344724b87f6
2024-06-19T21:41:31.3087180Z azure:core-rest-pipeline:info Request: {
2024-06-19T21:41:31.3087668Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token",
2024-06-19T21:41:31.3088129Z   "headers": {
2024-06-19T21:41:31.3088518Z     "accept": "application/json",
2024-06-19T21:41:31.3088996Z     "accept-encoding": "gzip,deflate",
2024-06-19T21:41:31.3089676Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.0 Node/17.9.1 OS/(x64-Windows_NT-10.0.20348)",
2024-06-19T21:41:31.3090378Z     "x-ms-client-request-id": "1f5ca017-909b-4b92-bcb2-0344724b87f6"
2024-06-19T21:41:31.3090773Z   },
2024-06-19T21:41:31.3091090Z   "method": "GET",
2024-06-19T21:41:31.3091398Z   "timeout": 1000,
2024-06-19T21:41:31.3091749Z   "disableKeepAlive": false,
2024-06-19T21:41:31.3092098Z   "withCredentials": false,
2024-06-19T21:41:31.3092435Z   "tracingOptions": {
2024-06-19T21:41:31.3092781Z     "tracingContext": {
2024-06-19T21:41:31.3093082Z       "_contextMap": {}
2024-06-19T21:41:31.3093397Z     }
2024-06-19T21:41:31.3093680Z   },
2024-06-19T21:41:31.3094220Z   "requestId": "1f5ca017-909b-4b92-bcb2-0344724b87f6",
2024-06-19T21:41:31.3094647Z   "allowInsecureConnection": true,
2024-06-19T21:41:31.3096099Z   "enableBrowserStreams": false
2024-06-19T21:41:31.3096455Z }
2024-06-19T21:41:31.3232819Z azure:core-rest-pipeline:info Response status code: 400
2024-06-19T21:41:31.3233441Z azure:core-rest-pipeline:info Headers: {
2024-06-19T21:41:31.3234109Z   "content-type": "application/json; charset=utf-8",
2024-06-19T21:41:31.3234564Z   "server": "IMDS/150.870.65.1278",
2024-06-19T21:41:31.3235106Z   "x-ms-request-id": "1ee96f2e-dd4c-4b8a-9882-10bd662cbbf6",
2024-06-19T21:41:31.3235551Z   "date": "Wed, 19 Jun 2024 21:41:30 GMT",
2024-06-19T21:41:31.3236023Z   "content-length": "88"
2024-06-19T21:41:31.3236348Z }
2024-06-19T21:41:31.3238586Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request 1f5ca017-909b-4b92-bcb2-0344724b87f6
2024-06-19T21:41:31.3239735Z azure:core-rest-pipeline retryPolicy:info Retry 0: Maximum retries reached. Returning the last received response, or throwing the last received error.
2024-06-19T21:41:31.3240734Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: The Azure IMDS endpoint is available
2024-06-19T21:41:31.3288315Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider invoked with parameters- {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-06-19T21:41:31.3289704Z azure:identity:info ManagedIdentityCredential => authenticateManagedIdentity invoked with scopes- ["499b84ac-1321-427f-aa17-267ca6975798/.default"] and getTokenOptions - {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-06-19T21:41:31.3291134Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Using the default Azure IMDS endpoint http://169.254.169.254.
2024-06-19T21:41:31.3296463Z azure:identity:info IdentityClient: sending token request to [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01]
2024-06-19T21:41:31.3299926Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request d033a9e0-e69a-4321-9b62-3ff969175e05
2024-06-19T21:41:31.3307349Z azure:core-rest-pipeline:info Request: {
2024-06-19T21:41:31.3308055Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token?resource=REDACTED&api-version=2018-02-01",
2024-06-19T21:41:31.3308555Z   "headers": {
2024-06-19T21:41:31.3308901Z     "accept": "application/json",
2024-06-19T21:41:31.3309246Z     "metadata": "REDACTED",
2024-06-19T21:41:31.3309737Z     "accept-encoding": "gzip,deflate",
2024-06-19T21:41:31.3310433Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.0 Node/17.9.1 OS/(x64-Windows_NT-10.0.20348)",
2024-06-19T21:41:31.3311155Z     "x-ms-client-request-id": "d033a9e0-e69a-4321-9b62-3ff969175e05"
2024-06-19T21:41:31.3311513Z   },
2024-06-19T21:41:31.3311825Z   "method": "GET",
2024-06-19T21:41:31.3312128Z   "timeout": 0,
2024-06-19T21:41:31.3312451Z   "disableKeepAlive": false,
2024-06-19T21:41:31.3312776Z   "withCredentials": false,
2024-06-19T21:41:31.3313328Z   "requestId": "d033a9e0-e69a-4321-9b62-3ff969175e05",
2024-06-19T21:41:31.3313602Z   "allowInsecureConnection": true,
2024-06-19T21:41:31.3313803Z   "enableBrowserStreams": false
2024-06-19T21:41:31.3313985Z }
2024-06-19T21:41:31.3894015Z azure:core-rest-pipeline:info Response status code: 200
2024-06-19T21:41:31.3897894Z azure:core-rest-pipeline:info Headers: {
2024-06-19T21:41:31.3900415Z   "content-type": "application/json; charset=utf-8",
2024-06-19T21:41:31.3901200Z   "server": "IMDS/150.870.65.1278",
2024-06-19T21:41:31.3901955Z   "x-ms-request-id": "79c31015-0204-499d-bb25-60724ba93adc",
2024-06-19T21:41:31.3904400Z   "date": "Wed, 19 Jun 2024 21:41:30 GMT",
2024-06-19T21:41:31.3905081Z   "content-length": "1532"
2024-06-19T21:41:31.3905530Z }
2024-06-19T21:41:31.3906334Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request d033a9e0-e69a-4321-9b62-3ff969175e05
2024-06-19T21:41:31.3910645Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing 2 retry strategies.
2024-06-19T21:41:31.3911689Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy throttlingRetryStrategy.
2024-06-19T21:41:31.3912712Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-06-19T21:41:31.3913718Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy exponentialRetryStrategy.
2024-06-19T21:41:31.3914652Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-06-19T21:41:31.3915692Z azure:core-rest-pipeline retryPolicy:info None of the retry strategies could work with the received response. Returning it.
2024-06-19T21:41:31.3917138Z azure:identity:info IdentityClient: [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01] token acquired, expires on 1718919691000
2024-06-19T21:41:31.3918152Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider will save the token in cache
2024-06-19T21:41:31.3934438Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-06-19T21:41:31.3935729Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-06-19T21:41:31.3937133Z azure:identity:info ChainedTokenCredential => getToken() => Result for ManagedIdentityCredential: SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-06-19T21:41:31.3961749Z  INFO  Publishing 'ms-dynamics-smb.al v14.0.1053825'...
2024-06-19T21:41:31.9107482Z  ERROR  {"$id":"1","innerException":null,"message":"The requested operation is not allowed.","typeName":"Microsoft.TeamFoundation.Framework.Server.InvalidAccessException, Microsoft.TeamFoundation.Framework.Server","typeKey":"InvalidAccessException","errorCode":0,"eventId":3000}
2024-06-19T21:41:31.9914983Z ##[error]Script failed with exit code: 1
2024-06-19T21:41:31.9930631Z [command]C:\Windows\system32\cmd.exe /D /S /C ""C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin\az.cmd" account clear"
2024-06-19T21:41:32.7734942Z ##[section]Finishing: Publish with managed identity
lszomoru commented 5 months ago

Based on the logs it looks like the ManagedIdentityCredential was used to get the token:

2024-06-19T21:41:31.3937133Z azure:identity:info ChainedTokenCredential => getToken() => Result for ManagedIdentityCredential: SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.

Are you using the Azure CLI task? Do you specify the addSpnToEnvironment: true for the task? I am asking as I seeing the following in the logs and I am wondering if some environment variables are not being set properly.

SBalslev commented 5 months ago

It is Azure Cli, PS Core. How do you specify addSpnToEnvironment: true? I don't see an option for it.

From: Ladislau Szomoru @.> Sent: Thursday, June 20, 2024 1:59:38 PM To: microsoft/vscode-vsce @.> Cc: Steffen Balslev @.>; Mention @.> Subject: Re: [microsoft/vscode-vsce] "The requested operation is not allowed" when using --azure-credential (Issue #976)

Based on the logs it looks like the ManagedIdentityCredential was used to get the token:

2024-06-19T21:41:31.3937133Z azure:identity:info ChainedTokenCredential => getToken() => Result for ManagedIdentityCredential: SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.

Are you using the Azure CLI taskhttps://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines? Do you specify the addSpnToEnvironment: true for the task? I am asking as I seeing the following in the logs and I am wondering if some environment variables are not being set properly.

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode-vsce/issues/976#issuecomment-2180496934, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACH3EK2FY5FVV6QQP3S6CFTZIK72VAVCNFSM6AAAAABHRI44L2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBQGQ4TMOJTGQ. You are receiving this because you were mentioned.Message ID: @.***>

lszomoru commented 5 months ago

See https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/azure-cli-v2?view=azure-pipelines#syntax.

SBalslev commented 5 months ago

Even with this set to true it fails. I compared the logs with the previous failure and the only difference is the request-id image

winstliu commented 5 months ago

I'll test with verbose logs on and get back to you; in the meantime, my hunch based on the above is that vsce is trying to use the managed identity on the VM (our pools all have one to stay in compliance), which is not the identity to use. We need the Az credentials provided by the AzureCLI wrapper task, which is lower priority on the DefaultAzureCredential chain.

As for addSpnToEnvironment: that adds environment variables in a different format than EnvironmentCredential expects (namely servicePrincipalId, tenantId, etc. rather than AZURE_CLIENT_ID, AZURE_TENANT_ID, etc.). It's also not clear how the translation would work when using WIF-enabled service connections which don't have a client secret.

All in all this seems tricky: I imagine there's certainly times when the managed identity should be used, and times when it shouldn't be.

winstliu commented 5 months ago

Yeah, confirmed it's trying to use the managed identity which is guaranteed to fail.

2024-06-26T23:03:11.7190763Z azure:identity:info EnvironmentCredential => Found the following environment variables: 
2024-06-26T23:03:11.7191611Z azure:identity:info WorkloadIdentityCredential => Found the following environment variables: 
2024-06-26T23:03:11.7195313Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-06-26T23:03:11.7209109Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-06-26T23:03:11.7231808Z azure:identity:info WorkloadIdentityCredential => WorkloadIdentityCredential: is unavailable. tenantId, clientId, and federatedTokenFilePath are required parameters. 
2024-06-26T23:03:11.7232414Z       In DefaultAzureCredential and ManagedIdentityCredential, these can be provided as environment variables - 
2024-06-26T23:03:11.7232878Z       "AZURE_TENANT_ID",
2024-06-26T23:03:11.7233184Z       "AZURE_CLIENT_ID",
2024-06-26T23:03:11.7233613Z       "AZURE_FEDERATED_TOKEN_FILE". See the troubleshooting guide for more information: https://aka.ms/azsdk/js/identity/workloadidentitycredential/troubleshoot  
2024-06-26T23:03:11.7241907Z azure:identity:info ManagedIdentityCredential - Azure Arc MSI => ManagedIdentityCredential - Azure Arc MSI: The environment variables needed are: IMDS_ENDPOINT and IDENTITY_ENDPOINT
2024-06-26T23:03:11.7248427Z azure:identity:info ManagedIdentityCredential - Fabric MSI => ManagedIdentityCredential - Fabric MSI: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT, IDENTITY_HEADER and IDENTITY_SERVER_THUMBPRINT
2024-06-26T23:03:11.7249587Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2019 => ManagedIdentityCredential - AppServiceMSI 2019: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT and IDENTITY_HEADER.
2024-06-26T23:03:11.7250564Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2017 => ManagedIdentityCredential - AppServiceMSI 2017: Unavailable. The environment variables needed are: MSI_ENDPOINT and MSI_SECRET.
2024-06-26T23:03:11.7251518Z azure:identity:info ManagedIdentityCredential - CloudShellMSI => ManagedIdentityCredential - CloudShellMSI: Unavailable. The environment variable MSI_ENDPOINT is needed.
2024-06-26T23:03:11.7252656Z azure:identity:info ManagedIdentityCredential - Token Exchange => ManagedIdentityCredential - Token Exchange: Unavailable. The environment variables needed are: AZURE_CLIENT_ID (or the client ID sent through the parameters), AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE
2024-06-26T23:03:11.7260893Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Pinging the Azure IMDS endpoint
2024-06-26T23:03:12.1256106Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request 85578d80-0fbf-4126-a5a0-6b106e696462
2024-06-26T23:03:12.1266008Z azure:core-rest-pipeline:info Request: {
2024-06-26T23:03:12.1266512Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token",
2024-06-26T23:03:12.1266929Z   "headers": {
2024-06-26T23:03:12.1267275Z     "accept": "application/json",
2024-06-26T23:03:12.1267732Z     "accept-encoding": "gzip,deflate",
2024-06-26T23:03:12.1268338Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.1 Node/18.20.3 OS/(x64-Windows_NT-10.0.20348)",
2024-06-26T23:03:12.1268982Z     "x-ms-client-request-id": "85578d80-0fbf-4126-a5a0-6b106e696462"
2024-06-26T23:03:12.1269358Z   },
2024-06-26T23:03:12.1269647Z   "method": "GET",
2024-06-26T23:03:12.1269946Z   "timeout": 1000,
2024-06-26T23:03:12.1270512Z   "disableKeepAlive": false,
2024-06-26T23:03:12.1270837Z   "withCredentials": false,
2024-06-26T23:03:12.1271137Z   "tracingOptions": {
2024-06-26T23:03:12.1271440Z     "tracingContext": {
2024-06-26T23:03:12.1271749Z       "_contextMap": {}
2024-06-26T23:03:12.1272027Z     }
2024-06-26T23:03:12.1273353Z   },
2024-06-26T23:03:12.1273932Z   "requestId": "85578d80-0fbf-4126-a5a0-6b106e696462",
2024-06-26T23:03:12.1274365Z   "allowInsecureConnection": true,
2024-06-26T23:03:12.1274707Z   "enableBrowserStreams": false
2024-06-26T23:03:12.1276098Z }
2024-06-26T23:03:12.1412088Z azure:core-rest-pipeline:info Response status code: 400
2024-06-26T23:03:12.1412610Z azure:core-rest-pipeline:info Headers: {
2024-06-26T23:03:12.1413169Z   "content-type": "application/json; charset=utf-8",
2024-06-26T23:03:12.1413535Z   "server": "IMDS/150.870.65.1305",
2024-06-26T23:03:12.1414020Z   "x-ms-request-id": "e2dfb8b0-1156-44a5-97a4-35d52e08e6f3",
2024-06-26T23:03:12.1414448Z   "date": "Wed, 26 Jun 2024 23:03:11 GMT",
2024-06-26T23:03:12.1414933Z   "content-length": "88"
2024-06-26T23:03:12.1415243Z }
2024-06-26T23:03:12.1417106Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request 85578d80-0fbf-4126-a5a0-6b106e696462
2024-06-26T23:03:12.1418043Z azure:core-rest-pipeline retryPolicy:info Retry 0: Maximum retries reached. Returning the last received response, or throwing the last received error.
2024-06-26T23:03:12.1418935Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: The Azure IMDS endpoint is available
2024-06-26T23:03:12.3483933Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider invoked with parameters- {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-06-26T23:03:12.3486692Z azure:identity:info ManagedIdentityCredential => authenticateManagedIdentity invoked with scopes- ["499b84ac-1321-427f-aa17-267ca6975798/.default"] and getTokenOptions - {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-06-26T23:03:12.3487863Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Using the default Azure IMDS endpoint http://169.254.169.254.
2024-06-26T23:03:12.3494851Z azure:identity:info IdentityClient: sending token request to [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01]
2024-06-26T23:03:12.3497375Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request f4d79147-d118-4cb6-9e58-fc15e3c51a40
2024-06-26T23:03:12.3505681Z azure:core-rest-pipeline:info Request: {
2024-06-26T23:03:12.3506337Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token?resource=REDACTED&api-version=2018-02-01",
2024-06-26T23:03:12.3506783Z   "headers": {
2024-06-26T23:03:12.3507117Z     "accept": "application/json",
2024-06-26T23:03:12.3507459Z     "metadata": "REDACTED",
2024-06-26T23:03:12.3507921Z     "accept-encoding": "gzip,deflate",
2024-06-26T23:03:12.3508568Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.1 Node/18.20.3 OS/(x64-Windows_NT-10.0.20348)",
2024-06-26T23:03:12.3509196Z     "x-ms-client-request-id": "f4d79147-d118-4cb6-9e58-fc15e3c51a40"
2024-06-26T23:03:12.3509568Z   },
2024-06-26T23:03:12.3509837Z   "method": "GET",
2024-06-26T23:03:12.3510118Z   "timeout": 0,
2024-06-26T23:03:12.3510410Z   "disableKeepAlive": false,
2024-06-26T23:03:12.3510691Z   "withCredentials": false,
2024-06-26T23:03:12.3511175Z   "requestId": "f4d79147-d118-4cb6-9e58-fc15e3c51a40",
2024-06-26T23:03:12.3511537Z   "allowInsecureConnection": true,
2024-06-26T23:03:12.3511877Z   "enableBrowserStreams": false
2024-06-26T23:03:12.3512250Z }
2024-06-26T23:03:12.6387147Z azure:core-rest-pipeline:info Response status code: 200
2024-06-26T23:03:12.6387741Z azure:core-rest-pipeline:info Headers: {
2024-06-26T23:03:12.6389793Z   "content-type": "application/json; charset=utf-8",
2024-06-26T23:03:12.6390324Z   "server": "IMDS/150.870.65.1305",
2024-06-26T23:03:12.6390871Z   "x-ms-request-id": "980b8d90-6863-4bea-bcad-dcb1465478fa",
2024-06-26T23:03:12.6391306Z   "date": "Wed, 26 Jun 2024 23:03:12 GMT",
2024-06-26T23:03:12.6393229Z   "content-length": "1534"
2024-06-26T23:03:12.6393542Z }
2024-06-26T23:03:12.6394118Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request f4d79147-d118-4cb6-9e58-fc15e3c51a40
2024-06-26T23:03:12.6394809Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing 2 retry strategies.
2024-06-26T23:03:12.6397545Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy throttlingRetryStrategy.
2024-06-26T23:03:12.6398278Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-06-26T23:03:12.6398917Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy exponentialRetryStrategy.
2024-06-26T23:03:12.6399466Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-06-26T23:03:12.6400097Z azure:core-rest-pipeline retryPolicy:info None of the retry strategies could work with the received response. Returning it.
2024-06-26T23:03:12.6400942Z azure:identity:info IdentityClient: [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01] token acquired, expires on 1719529392000
2024-06-26T23:03:12.6401580Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider will save the token in cache
2024-06-26T23:03:12.6419488Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-06-26T23:03:12.6420284Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-06-26T23:03:12.6421208Z azure:identity:info ChainedTokenCredential => getToken() => Result for ManagedIdentityCredential: SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.

After thinking about this a bit more...I think it might make sense for vsce to have a separate argument controlling whether a managed identity is used, and for --azure-credential to only be for user auth. My reasoning is that pools can have multiple managed identities attached to them, in which case DefaultAzureCredential will also fail because it won't know which MI to choose.

By providing a dedicated argument, e.g. --managed-identity, people who want to use MIs can specify the client ID of the one they want. This is similar to how other tools do it, such as the Git Credential Manager.

Then --azure-credential can be repurposed to only accept user credentials.

lszomoru commented 5 months ago

As for addSpnToEnvironment: that adds environment variables in a different format than EnvironmentCredential expects (namely servicePrincipalId, tenantId, etc. rather than AZURE_CLIENT_ID, AZURE_TENANT_ID, etc.). It's also not clear how the translation would work when using WIF-enabled service connections which don't have a client secret.

Looking at the Azure CLI task's source code, it looks like when you are using WIF the idToken environment variable is set.

I am not yet convinced that having different flags for different auth methods is the way to go. The nice thing with the current implementation is that it follows the default auth chain that can be controlled using environment variables (lots of good documentation and sample available).

I will be doing some more testing tomorrow and circle back to this issue with my findings. When you set up your service connection that uses workload identity federation did you use the "automatic" flow, or the "manual" flow?

lszomoru commented 4 months ago

I have set up a sample pipeline and used the following task successfully both with a service connection that was created using the "automatic" flow as well as the "manual" flow. If you still have problems getting this to work I would love to see the output that is being displayed with your pipeline (please replace <SERVICE_CONNECTION> with your own service connection and <PUBLISHER_NAME> with your own publisher):

steps:
- task: AzureCLI@2
  displayName: 'Azure CLI'
  inputs:
    azureSubscription: <SERVICE_CONNECTION>
    scriptType: ps
    scriptLocation: inlineScript
    inlineScript: |
     $env:AZURE_LOG_LEVEL = "verbose"

     az rest -u https://app.vssps.visualstudio.com/_apis/profile/profiles/me --resource 499b84ac-1321-427f-aa17-267ca6975798
     npx @vscode/vsce@latest verify-pat <PUBLISHER_NAME> --azure-credential
winstliu commented 4 months ago

@lszomoru does your pool have a managed identity attached to it? Here is what I see with the requested changes.

2024-07-01T18:37:20.2330611Z {
2024-07-01T18:37:20.2383089Z   "coreRevision": 453058499,
2024-07-01T18:37:20.2493911Z   "displayName": "72f988bf-86f1-41af-91ab-2d7cd011db47\\166e71d4-3590-4a4c-9bb8-cc51e2cd3e5e",
2024-07-01T18:37:20.2531786Z   "emailAddress": "",
2024-07-01T18:37:20.2533404Z   "id": "3c15128d-adc3-6659-a6e0-99ea01e6e33d",
2024-07-01T18:37:20.2824824Z   "publicAlias": "3c15128d-adc3-6659-a6e0-99ea01e6e33d",
2024-07-01T18:37:20.2827748Z   "revision": 453058499,
2024-07-01T18:37:20.2853044Z   "timeStamp": "2024-05-10T17:34:59.6933333+00:00"
2024-07-01T18:37:20.2862802Z }
2024-07-01T18:37:21.4984701Z azure:identity:info EnvironmentCredential => Found the following environment variables: 
2024-07-01T18:37:21.5025826Z azure:identity:info WorkloadIdentityCredential => Found the following environment variables: 
2024-07-01T18:37:21.5047809Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-07-01T18:37:21.5051820Z azure:core-client:warning The baseUri option for SDK Clients has been deprecated, please use endpoint instead.
2024-07-01T18:37:21.5053068Z azure:identity:info WorkloadIdentityCredential => WorkloadIdentityCredential: is unavailable. tenantId, clientId, and federatedTokenFilePath are required parameters. 
2024-07-01T18:37:21.5053729Z       In DefaultAzureCredential and ManagedIdentityCredential, these can be provided as environment variables - 
2024-07-01T18:37:21.5054104Z       "AZURE_TENANT_ID",
2024-07-01T18:37:21.5054399Z       "AZURE_CLIENT_ID",
2024-07-01T18:37:21.5054859Z       "AZURE_FEDERATED_TOKEN_FILE". See the troubleshooting guide for more information: https://aka.ms/azsdk/js/identity/workloadidentitycredential/troubleshoot  
2024-07-01T18:37:21.5055712Z azure:identity:info ManagedIdentityCredential - Azure Arc MSI => ManagedIdentityCredential - Azure Arc MSI: The environment variables needed are: IMDS_ENDPOINT and IDENTITY_ENDPOINT
2024-07-01T18:37:21.5056645Z azure:identity:info ManagedIdentityCredential - Fabric MSI => ManagedIdentityCredential - Fabric MSI: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT, IDENTITY_HEADER and IDENTITY_SERVER_THUMBPRINT
2024-07-01T18:37:21.5057586Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2019 => ManagedIdentityCredential - AppServiceMSI 2019: Unavailable. The environment variables needed are: IDENTITY_ENDPOINT and IDENTITY_HEADER.
2024-07-01T18:37:21.5058463Z azure:identity:info ManagedIdentityCredential - AppServiceMSI 2017 => ManagedIdentityCredential - AppServiceMSI 2017: Unavailable. The environment variables needed are: MSI_ENDPOINT and MSI_SECRET.
2024-07-01T18:37:21.5059292Z azure:identity:info ManagedIdentityCredential - CloudShellMSI => ManagedIdentityCredential - CloudShellMSI: Unavailable. The environment variable MSI_ENDPOINT is needed.
2024-07-01T18:37:21.5060277Z azure:identity:info ManagedIdentityCredential - Token Exchange => ManagedIdentityCredential - Token Exchange: Unavailable. The environment variables needed are: AZURE_CLIENT_ID (or the client ID sent through the parameters), AZURE_TENANT_ID and AZURE_FEDERATED_TOKEN_FILE
2024-07-01T18:37:21.5061141Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Pinging the Azure IMDS endpoint
2024-07-01T18:37:23.4653745Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request c170f528-2f09-44bc-b769-eefc43137956
2024-07-01T18:37:23.4661038Z azure:core-rest-pipeline:info Request: {
2024-07-01T18:37:23.4661448Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token",
2024-07-01T18:37:23.4662062Z   "headers": {
2024-07-01T18:37:23.4662400Z     "accept": "application/json",
2024-07-01T18:37:23.4662833Z     "accept-encoding": "gzip,deflate",
2024-07-01T18:37:23.4663411Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.1 Node/18.20.3 OS/(x64-Windows_NT-10.0.20348)",
2024-07-01T18:37:23.4665121Z     "x-ms-client-request-id": "c170f528-2f09-44bc-b769-eefc43137956"
2024-07-01T18:37:23.4665497Z   },
2024-07-01T18:37:23.4665808Z   "method": "GET",
2024-07-01T18:37:23.4666096Z   "timeout": 1000,
2024-07-01T18:37:23.4666385Z   "disableKeepAlive": false,
2024-07-01T18:37:23.4666709Z   "withCredentials": false,
2024-07-01T18:37:23.4667009Z   "tracingOptions": {
2024-07-01T18:37:23.4667326Z     "tracingContext": {
2024-07-01T18:37:23.4667622Z       "_contextMap": {}
2024-07-01T18:37:23.4667919Z     }
2024-07-01T18:37:23.4668190Z   },
2024-07-01T18:37:23.4668660Z   "requestId": "c170f528-2f09-44bc-b769-eefc43137956",
2024-07-01T18:37:23.4669091Z   "allowInsecureConnection": true,
2024-07-01T18:37:23.4669453Z   "enableBrowserStreams": false
2024-07-01T18:37:23.4669778Z }
2024-07-01T18:37:23.4787444Z azure:core-rest-pipeline:info Response status code: 400
2024-07-01T18:37:23.4788062Z azure:core-rest-pipeline:info Headers: {
2024-07-01T18:37:23.4788604Z   "content-type": "application/json; charset=utf-8",
2024-07-01T18:37:23.4791543Z   "server": "IMDS/150.870.65.1305",
2024-07-01T18:37:23.4792129Z   "x-ms-request-id": "7e77850e-a6a5-436b-844f-36fb2da56311",
2024-07-01T18:37:23.4792535Z   "date": "Mon, 01 Jul 2024 18:37:23 GMT",
2024-07-01T18:37:23.4792943Z   "content-length": "88"
2024-07-01T18:37:23.4793264Z }
2024-07-01T18:37:23.4793929Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request c170f528-2f09-44bc-b769-eefc43137956
2024-07-01T18:37:23.4795939Z azure:core-rest-pipeline retryPolicy:info Retry 0: Maximum retries reached. Returning the last received response, or throwing the last received error.
2024-07-01T18:37:23.4796906Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: The Azure IMDS endpoint is available
2024-07-01T18:37:23.6873417Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider invoked with parameters- {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-07-01T18:37:23.6876125Z azure:identity:info ManagedIdentityCredential => authenticateManagedIdentity invoked with scopes- ["499b84ac-1321-427f-aa17-267ca6975798/.default"] and getTokenOptions - {"correlationId":"noCorrelationId","tenantId":"managed_identity","scopes":["499b84ac-1321-427f-aa17-267ca6975798/.default"]}
2024-07-01T18:37:23.6877201Z azure:identity:info ManagedIdentityCredential - IMDS => ManagedIdentityCredential - IMDS: Using the default Azure IMDS endpoint http://169.254.169.254.
2024-07-01T18:37:23.6884253Z azure:identity:info IdentityClient: sending token request to [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01]
2024-07-01T18:37:23.6886708Z azure:core-rest-pipeline retryPolicy:info Retry 0: Attempting to send request 1724dbef-7619-448f-b494-95a576f8f405
2024-07-01T18:37:23.6894772Z azure:core-rest-pipeline:info Request: {
2024-07-01T18:37:23.6895405Z   "url": "http://169.254.169.254/metadata/identity/oauth2/token?resource=REDACTED&api-version=2018-02-01",
2024-07-01T18:37:23.6895856Z   "headers": {
2024-07-01T18:37:23.6896201Z     "accept": "application/json",
2024-07-01T18:37:23.6896515Z     "metadata": "REDACTED",
2024-07-01T18:37:23.6896981Z     "accept-encoding": "gzip,deflate",
2024-07-01T18:37:23.6897595Z     "user-agent": "azsdk-js-identity/4.3.0-beta.3 core-rest-pipeline/1.16.1 Node/18.20.3 OS/(x64-Windows_NT-10.0.20348)",
2024-07-01T18:37:23.6898226Z     "x-ms-client-request-id": "1724dbef-7619-448f-b494-95a576f8f405"
2024-07-01T18:37:23.6898782Z   },
2024-07-01T18:37:23.6899052Z   "method": "GET",
2024-07-01T18:37:23.6899321Z   "timeout": 0,
2024-07-01T18:37:23.6899584Z   "disableKeepAlive": false,
2024-07-01T18:37:23.6899886Z   "withCredentials": false,
2024-07-01T18:37:23.6900360Z   "requestId": "1724dbef-7619-448f-b494-95a576f8f405",
2024-07-01T18:37:23.6900703Z   "allowInsecureConnection": true,
2024-07-01T18:37:23.6901188Z   "enableBrowserStreams": false
2024-07-01T18:37:23.6901455Z }
2024-07-01T18:37:23.7462931Z azure:core-rest-pipeline:info Response status code: 200
2024-07-01T18:37:23.7465973Z azure:core-rest-pipeline:info Headers: {
2024-07-01T18:37:23.7466545Z   "content-type": "application/json; charset=utf-8",
2024-07-01T18:37:23.7468229Z   "server": "IMDS/150.870.65.1305",
2024-07-01T18:37:23.7468842Z   "x-ms-request-id": "4f1bb65b-cfce-4c22-a738-e29402989541",
2024-07-01T18:37:23.7469278Z   "date": "Mon, 01 Jul 2024 18:37:23 GMT",
2024-07-01T18:37:23.7471732Z   "content-length": "1532"
2024-07-01T18:37:23.7472106Z }
2024-07-01T18:37:23.7472756Z azure:core-rest-pipeline retryPolicy:info Retry 0: Received a response from request 1724dbef-7619-448f-b494-95a576f8f405
2024-07-01T18:37:23.7477461Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing 2 retry strategies.
2024-07-01T18:37:23.7478115Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy throttlingRetryStrategy.
2024-07-01T18:37:23.7478732Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-07-01T18:37:23.7479392Z azure:core-rest-pipeline retryPolicy:info Retry 0: Processing retry strategy exponentialRetryStrategy.
2024-07-01T18:37:23.7479995Z azure:core-rest-pipeline retryPolicy:info Retry 0: Skipped.
2024-07-01T18:37:23.7480568Z azure:core-rest-pipeline retryPolicy:info None of the retry strategies could work with the received response. Returning it.
2024-07-01T18:37:23.7481464Z azure:identity:info IdentityClient: [http://169.254.169.254/metadata/identity/oauth2/token?resource=499b84ac-1321-427f-aa17-267ca6975798&api-version=2018-02-01] token acquired, expires on 1719945443000
2024-07-01T18:37:23.7482086Z azure:identity:info ManagedIdentityCredential => SetAppTokenProvider will save the token in cache
2024-07-01T18:37:23.7493332Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-07-01T18:37:23.7494165Z azure:identity:info ManagedIdentityCredential => getToken() => SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-07-01T18:37:23.7494983Z azure:identity:info ChainedTokenCredential => getToken() => Result for ManagedIdentityCredential: SUCCESS. Scopes: 499b84ac-1321-427f-aa17-267ca6975798/.default.
2024-07-01T18:37:23.7515108Z  INFO  Publishing 'redacted.redacted v1.2024.701003'...
2024-07-01T18:37:24.2696395Z  ERROR  {"$id":"1","innerException":null,"message":"The requested operation is not allowed.","typeName":"Microsoft.TeamFoundation.Framework.Server.InvalidAccessException, Microsoft.TeamFoundation.Framework.Server","typeKey":"InvalidAccessException","errorCode":0,"eventId":3000}

Client ID 166e71d4-3590-4a4c-9bb8-cc51e2cd3e5e as reported by the az rest command is the correct identity we're trying to publish under.

In regards to your previous comments...

The nice thing with the current implementation is that it follows the default auth chain that can be controlled using environment variables (lots of good documentation and sample available).

True, but for someone who just wants to publish an extension, having to know internal implementation details such as which auth library vsce is using and how to control that implementation doesn't seem like a good user experience. None of this is currently documented in the vsce README/CLI help or the Publishing Extensions page, and the current error message only says "The requested operation is not allowed".

When you set up your service connection that uses workload identity federation did you use the "automatic" flow, or the "manual" flow?

Manual.

From these logs and from your mention of the idToken variable, it seems like I can get around this by writing idToken to a file, and then passing in that file as AZURE_FEDERATED_TOKEN_FILE. But that really seems quite insecure, as now I'd need to write a secret to disk.

lszomoru commented 4 months ago

Spent some more time looking at this and indeed the issue is a combination of the authentication order, and the fact that there is a managed identity associated with the agent pool. If there is no managed identity associated with the agent pool things work out of the box. If you do have a managed identity associated with the agent pool the recommended workaround is to use the following code:

$env:AZURE_TENANT_ID = "00000000-0000-0000-0000-000000000000"
$env:AZURE_CLIENT_ID = "00000000-0000-0000-0000-000000000000"

npx @vscode/vsce@latest verify-pat <PUBLISHER_NAME> --azure-credential

By "clearing" the two environment variables we ensure that the ManagedIdentityCredential is not created and the AzureCLICredential is created based on the service connection. Please give it a try and let me know if that works for you.

I am also looking at updating vsce to use a ChainedTokenCredential that would swap the order of the AzureCLICredential and ManagedIdentityCredential as that should cover the various scenarios in which vsce is being used within an Azure Pipeline.

lszomoru commented 4 months ago

I have pushed a pre-release version of vsce that uses a custom ChainedTokenCredential where the order of the AzureCLICredential and ManagedIdentityCredential are swapped. This means that if you run vsce --azure-credential within an AzureCLI task we will always use the AzureCLICredential even if you have a managed identity associated with the build agent. If you want vsce --azure-identity to use a managed identity that is associated with the build agent then you have to run vsce using a scripttask.

Could you please try the following code and let me know if it's working as expected:

npx @vscode/vsce@next verify-pat <PUBLISHER_NAME> --azure-credential

If everything goes as expected we will publish a stable version of vsce that will contain these changes. Thank you!

winstliu commented 4 months ago

By "clearing" the two environment variables

This worked, thanks!

I will also try the pre-release version and report back :)

winstliu commented 4 months ago

I will also try the pre-release version and report back :)

This also worked! We removed the AZURE_CLIENT_ID workaround and were still able to publish.

trevors20 commented 4 months ago 
$env:AZURE_TENANT_ID = "00000000-0000-0000-0000-000000000000"
$env:AZURE_CLIENT_ID = "00000000-0000-0000-0000-000000000000"

npx @vscode/vsce@latest verify-pat <PUBLISHER_NAME> --azure-credential

By "clearing" the two environment variables we ensure that the ManagedIdentityCredential is not created and the AzureCLICredential is created based on the service connection. Please give it a try and let me know if that works for you.

FYI. I was unable to get this to work unless I did not zero out the AZURE_TENANT_ID environment variable. When I tried this I got the following error: CredentialUnavailableError: Error: Get-AzAccessToken: ClientAssertionCredential authentication failed: AADSTS900021: Requested tenant identifier '00000000-0000-0000-0000-000000000000' is not valid. Tenant identifiers may not be an empty GUID. Trace ID: 97acb6e8-4d95-41ad-8381-eccb67dc2801 Correlation ID: 8a25e179-11e2-4a0c-a5bb-c6f734889b30 Timestamp: 2024-07-12 18:04:55Z

I had to comment out the $env:AZURE_TENANT_ID line and it worked after tha. Not sure if it is related or not, but the AzurePowerShell Task (which I am using) was recently updated to stop using AzureRM modules and to use Az modules instead. Could be related.

Is there a timeline for when the pre-release changes that you have will be deployed to production? Can't wait! And, thank you for working on this.

@lszomoru

naiyunzhang commented 4 months ago

I've tried version 2.30.0, and it works for me. Thank you all. I really appreciate it.

lszomoru commented 4 months ago

We have published vsce version 2.30.0 that contains the fix for this issue. Please give it a try and let me know if you run into any issues. Thank you all!

trevors20 commented 4 months ago

@lszomoru , I just tried this out to publish something today. But, unfortunately, I ran into a weird error that I'm not sure what is going on. It looks like this Issue was closed down so I'll open a new issue for what I'm seeing. Thanks for your help!