devops extension does not work with MSI

Describe the bug

We have an internal deployment system which spins up azure container instance. When ACI is spun up with the release pipeline, azure devops extension does not work with MSI

We get this error when we try to access devops commands.

az login --identity 
aud="499b84ac-1321-427f-aa17-267ca6975798"
token=$(az account get-access-token --resource=$aud --query accessToken --output tsv)
echo "$token" |  az devops login --organization https://dev.azure.com/myorganization
az devops configure -d organization=https://dev.azure.com/myorganization project=myproject
az repos list -o table | head
ERROR: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.

If we spin up an ACI with the exact same image manually, it works fine.

Can you help?

Related command

echo "$token" |  az devops login --organization https://dev.azure.com/myorganization
az devops configure -d organization=https://dev.azure.com/myorganization project=myproject
az repos list -o table | head

Errors

ERROR: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.

Issue script & Debug output

DEBUG: cli.knack.cli: Command arguments: ['repos', 'list', '-o', 'table', '--debug'] DEBUG: cli.knack.cli: init debug log: Cannot enable color. DEBUG: cli.knack.cli: Event: Cli.PreExecute [] DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7fc49b0b8040>, <function OutputProducer.on_global_arguments at 0x7fc49b062160>, <function CLIQuery.on_global_arguments at 0x7fc49b09fc40>] DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate [] DEBUG: cli.azure.cli.core: Modules found from index for 'repos': ['azext_devops'] DEBUG: cli.azure.cli.core: Loading command modules: DEBUG: cli.azure.cli.core: Name Load Time Groups Commands DEBUG: cli.azure.cli.core: Total (0) 0.000 0 0 DEBUG: cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next'] DEBUG: cli.azure.cli.core: Loading extensions: DEBUG: cli.azure.cli.core: Name Load Time Groups Commands Directory DEBUG: cli.azure.cli.core: azure-devops 0.062 60 192 /root/.azure/cliextensions/azure-devops DEBUG: cli.azure.cli.core: Total (1) 0.062 60 192 DEBUG: cli.azure.cli.core: Loaded 60 groups, 192 commands. DEBUG: cli.azure.cli.core: Found a match in the command table. DEBUG: cli.azure.cli.core: Raw command : repos list DEBUG: cli.azure.cli.core: Command table: repos list DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7fc49a361c60>] DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-06-26.02-48-29.repos_list.341.log'. INFO: az_command_data_logger: command args: repos list -o {} --debug DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7fc49a156d40>] DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad [] DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7fc49a189f80>, <function register_cache_arguments..add_cache_arguments at 0x7fc49a18a0c0>] DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded [] DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [] DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7fc49b062200>, <function CLIQuery.handle_query_parameter at 0x7fc49b09fce0>, <function register_ids_argument..parse_ids_arguments at 0x7fc49a18a020>, <function DevCommandsLoader.post_parse_args at 0x7fc49a1c8900>] INFO: az_command_data_logger: extension name: azure-devops INFO: az_command_data_logger: extension version: 1.0.1 INFO: cli.azext_devops.dev.common.git: GitDetect: Could not detect current remotes based on current working directory. DEBUG: cli.azext_devops.dev.common.git: [Errno 2] No such file or directory: 'git' Traceback (most recent call last): File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/common/git.py", line 106, in get_git_remotes output = subprocess.check_output([_GIT_EXE, 'remote', '-v'], stderr=subprocess.STDOUT) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/subprocess.py", line 466, in check_output return run(popenargs, stdout=PIPE, timeout=timeout, check=True, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/subprocess.py", line 548, in run with Popen(popenargs, **kwargs) as process: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/subprocess.py", line 1026, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/opt/az/lib/python3.11/subprocess.py", line 1953, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'git' INFO: cli.azext_devops.dev.common.services: Detect: Url discovery took 0:00:00.001583 DEBUG: cli.azext_devops.dev.common.services: PAT is present which can be used against this instance DEBUG: cli.azure.cli.core: Current cloud config: AzureCloud DEBUG: cli.azext_devops.dev.common.services: trying to get token (temp) for tenant 72f988bf-86f1-41af-91ab-2d7cd011db47 and user systemAssignedIdentity DEBUG: cli.azext_devops.dev.common.services: not able to get token from az login DEBUG: cli.azext_devops.dev.common.services: Tenant shouldn't be specified for managed identity account Traceback (most recent call last): File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/common/services.py", line 157, in get_token_from_az_login raw = profile.get_raw_token( ^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/_profile.py", line 368, in get_raw_token raise CLIError("Tenant shouldn't be specified for managed identity account") knack.util.CLIError: Tenant shouldn't be specified for managed identity account DEBUG: cli.azext_devops.dev.common.services: instance recieved in validate_token_for_instance https://dev.azure.com/msazure DEBUG: cli.azext_devops.dev.common.services: instance processed in validate_token_for_instance https://dev.azure.com/msazure DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: azext_devops.devops_sdk._file_cache: Loading cache file: /root/.azure-devops/python-sdk/cache/resources.json DEBUG: azext_devops.devops_sdk._file_cache: attempting to read file /root/.azure-devops/python-sdk/cache/resources.json as utf-8-sig DEBUG: azext_devops.devops_sdk.connection: File cache hit for resources on: https://dev.azure.com/msazure DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: azext_devops.devops_sdk._file_cache: Loading cache file: /root/.azure-devops/python-sdk/cache/options.json DEBUG: azext_devops.devops_sdk._file_cache: attempting to read file /root/.azure-devops/python-sdk/cache/options.json as utf-8-sig DEBUG: azext_devops.devops_sdk.client: File cache miss for options on: https://msazure.visualstudio.com DEBUG: azext_devops.devops_sdk.client: OPTIONS https://msazure.visualstudio.com/_apis DEBUG: msrest.universal_http: Configuring redirects: allow=True, max=30 DEBUG: msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None DEBUG: msrest.universal_http: Configuring proxies: '' DEBUG: msrest.universal_http: Evaluate proxies against ENV settings: True DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): msazure.visualstudio.com:443 DEBUG: urllib3.connectionpool: https://msazure.visualstudio.com:443 "OPTIONS /_apis HTTP/1.1" 401 343 DEBUG: azext_devops.devops_sdk.client: Response content: b'{"$id":"1","innerException":null,"message":"TF400813: The user \'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa\' is not authorized to access this resource.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000}' DEBUG: msrest.exceptions: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. DEBUG: cli.azext_devops.dev.common.services: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. Traceback (most recent call last): File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/common/services.py", line 93, in validate_token_for_instance core_client.get_projects(state_filter='all', top=1, skip=0) File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/v5_0/core/core_client.py", line 140, in get_projects response = self._end(http_method='GET', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 60, in _send request = self._create_request_message(http_method=http_method, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 105, in _create_request_message location = self._get_resource_location(location_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 135, in _get_resource_location Client._locations_cache[self.config.base_url] = self._get_resource_locations(all_host_types=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 171, in _get_resource_locations response = self._send_request(request, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 54, in _send_request self._handle_error(request, response) File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 233, in _handle_error raise AzureDevOpsServiceError(wrapped_exception) azext_devops.devops_sdk.exceptions.AzureDevOpsServiceError: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. DEBUG: cli.azext_devops.dev.common.services: Failed to connect using provided credentials DEBUG: cli.azext_devops.dev.common.services: invalid token obtained for tenant 72f988bf-86f1-41af-91ab-2d7cd011db47 INFO: cli.azext_devops.dev.common.services: received PAT from environment variable INFO: cli.azext_devops.dev.common.services: Creating connection with personal access token. DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: cli.azext_devops.dev.common.telemetry: Azure devops telemetry enabled. DEBUG: cli.azext_devops.dev.common.telemetry: Logging telemetry to azure devops server. DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: azext_devops.devops_sdk.connection: File cache hit for resources on: https://dev.azure.com/msazure DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: azext_devops.devops_sdk.client: File cache hit for options on: https://dev.azure.com/msazure DEBUG: msrest.universal_http.requests: Configuring retry: max_retries=3, backoff_factor=0.8, max_backoff=90 DEBUG: azext_devops.devops_sdk.client: File cache miss for options on: https://msazure.visualstudio.com DEBUG: azext_devops.devops_sdk.client: OPTIONS https://msazure.visualstudio.com/_apis DEBUG: msrest.universal_http: Configuring redirects: allow=True, max=30 DEBUG: msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None DEBUG: msrest.universal_http: Configuring proxies: '' DEBUG: msrest.universal_http: Evaluate proxies against ENV settings: True DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): msazure.visualstudio.com:443 DEBUG: azext_devops.devops_sdk.client: Route template: _apis/{area}/{resource} DEBUG: azext_devops.devops_sdk.client: Api version '5.0-preview.1' DEBUG: azext_devops.devops_sdk.client: POST https://dev.azure.com/msazure/_apis/CustomerIntelligence/Events DEBUG: azext_devops.devops_sdk.client: Request content: [{'area': 'AzureDevopsCli', 'feature': 'repos', 'properties': {'Command': 'list', 'Args': '', 'ShellType': None, 'IsInteractive': 'True', 'OutputType': 'table', 'OrgPresentInCommand': False, 'ProjectPresentInCommand': False, 'RepoPresentInCommand': False, 'OrgPickedFromGit': False, 'ProjectPickedFromGit': False, 'RepoPickedFromGit': False, 'OrgPickedFromConfig': True, 'ProjectPickedFromConfig': True}} DEBUG: msrest.universal_http: Configuring redirects: allow=True, max=30 DEBUG: msrest.universal_http: Configuring request: timeout=100, verify=True, cert=None DEBUG: msrest.universal_http: Configuring proxies: '' DEBUG: msrest.universal_http: Evaluate proxies against ENV settings: True DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): dev.azure.com:443 DEBUG: urllib3.connectionpool: https://msazure.visualstudio.com:443 "OPTIONS /_apis HTTP/1.1" 401 343 DEBUG: azext_devops.devops_sdk.client: Response content: b'{"$id":"1","innerException":null,"message":"TF400813: The user \'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa\' is not authorized to access this resource.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000}' DEBUG: msrest.exceptions: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. DEBUG: cli.azext_devops.dev.common.exception_handler: handling vsts service error DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last): File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 701, in _run_job result = cmd_copy(params) ^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 334, in call return self.handler(*args, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler return op(command_args) ^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/repos/repository.py", line 64, in list_repos repository = git_client.get_repositories(project=project) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/v5_0/git/git_client_base.py", line 1995, in get_repositories response = self._send(http_method='GET', ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 60, in _send request = self._create_request_message(http_method=http_method, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 105, in _create_request_message location = self._get_resource_location(location_id) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 135, in _get_resource_location Client._locations_cache[self.config.base_url] = self._get_resource_locations(all_host_types=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 171, in _get_resource_locations response = self._send_request(request, headers=headers) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 54, in _send_request self._handle_error(request, response) File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 233, in _handle_error raise AzureDevOpsServiceError(wrapped_exception) azext_devops.devops_sdk.exceptions.AzureDevOpsServiceError: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/opt/az/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke cmd_result = self.invocation.execute(args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 664, in execute raise ex File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 731, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/az/lib/python3.11/site-packages/azure/cli/core/commands/init.py", line 723, in _run_job return cmd_copy.exception_handler(ex) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/common/exception_handler.py", line 18, in azure_devops_exception_handler raise CLIError(ex) knack.util.CLIError: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource.

ERROR: cli.azure.cli.core.azclierror: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. ERROR: az_command_data_logger: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fc49a361ee0>] INFO: az_command_data_logger: exit code: 1 INFO: cli.main: Command ran in 0.663 seconds (init: 0.156, invoke: 0.506) DEBUG: urllib3.connectionpool: https://dev.azure.com:443 "POST /msazure/_apis/CustomerIntelligence/Events HTTP/1.1" 401 343 DEBUG: azext_devops.devops_sdk.client: Response content: b'{"$id":"1","innerException":null,"message":"TF400813: The user \'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa\' is not authorized to access this resource.","typeName":"Microsoft.TeamFoundation.Framework.Server.UnauthorizedRequestException, Microsoft.TeamFoundation.Framework.Server","typeKey":"UnauthorizedRequestException","errorCode":0,"eventId":3000}' DEBUG: msrest.exceptions: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. DEBUG: cli.azext_devops.dev.common.telemetry: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. Traceback (most recent call last): File "/root/.azure/cliextensions/azure-devops/azext_devops/dev/common/telemetry.py", line 86, in _send_tracking_ci_event ci_client.publish_events([vsts_tracking_data]) File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/v5_0/customer_intelligence/customer_intelligence_client.py", line 20, in publish_events self._send(http_method='POST', File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 90, in _send response = self._send_request(request=request, headers=headers, content=content, media_type=media_type) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 54, in _send_request self._handle_error(request, response) File "/root/.azure/cliextensions/azure-devops/azext_devops/devops_sdk/client.py", line 233, in _handle_error raise AzureDevOpsServiceError(wrapped_exception) azext_devops.devops_sdk.exceptions.AzureDevOpsServiceError: TF400813: The user 'aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa' is not authorized to access this resource. INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1 INFO: telemetry.client: Accumulated 0 events. Flush the clients. INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1 INFO: telemetry.save: Save telemetry record of length 3780 in cache INFO: telemetry.main: Begin creating telemetry upload process. INFO: telemetry.process: Creating upload process: "/opt/az/bin/python3 /opt/az/lib/python3.11/site-packages/azure/cli/telemetry/init.py /root/.azure" INFO: telemetry.process: Return from creating process INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

A list of repositories should be shown.

Environment Summary

root@SandboxHost-638549630646745138:/# az version { "azure-cli": "2.59.0", "azure-cli-core": "2.59.0", "azure-cli-telemetry": "1.1.0", "extensions": { "azure-devops": "1.0.1" } }

Additional context

N/A

Azure / azure-cli