az dt route list Crashes on execution

MatthijsvdVeer commented 4 years ago

Describe the bug

When running the command az dt route list -m <adt-instance-name> I get the below error. Other az dt commands are working.

Command Name az dt route list Extension Name: azure-iot. Version: 0.9.7.

Errors:

400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 654, in execute
    raise ex
cli/core/commands/__init__.py, ln 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 711, in _run_job
    six.reraise(*sys.exc_info())
...
python3.6/site-packages/msrestazure/azure_active_directory.py, ln 486, in get_msi_token
    result.raise_for_status()
python3.6/site-packages/requests/models.py, ln 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token

To Reproduce:

Steps to reproduce the behaviour. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...

az dt route list -n {adt-instance-name} -g {resource group} --debug This is the debug ouput (some data redacted):


Command arguments: ['dt', 'route', 'list', '-g', '{REDACTED}', '-n', '{REDACTED}', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f159d9e7ea0>, <function OutputProducer.on_global_arguments at 0x7f159d7beea0>, <function CLIQuery.on_global_arguments at 0x7f159d565158>]
Event: CommandInvoker.OnPreCommandTableCreate []
Modules found from index for 'dt': ['azext_iot']
Loading command modules:
Name                  Load Time    Groups  Commands
Total (0)                 0.000         0         0
These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_ai_did_you_mean_this']
Loading extensions:
Name                  Load Time    Groups  Commands  Directory
azure-iot                 0.231        42       141  /home/{REDACTED}/.azure/cliextensions/azure-iot
Total (1)                 0.231        42       141
Loaded 42 groups, 141 commands.
Found a match in the command table for 'dt route list'
Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f159cf1ebf8>]
az_command_data_logger : command args: dt route list -g {} -n {} --debug
metadata file logging enabled - writing logs to '/home/{REDACTED}/.azure/commands'.
Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f159cf3ed90>]
Event: CommandInvoker.OnPostArgumentLoad []
Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at0x7f159cef1ea0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f159cf08048>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs []
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f159d7bef28>, <function CLIQuery.handle_query_parameter at 0x7f159d5651e0>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f159cef1f28>]
az_command_data_logger : extension name: azure-iot
az_command_data_logger : extension version: 0.9.7
Command group 'dt' is in preview. It may be changed/removed in a future release.
Getting management service client client_type=AzureDigitalTwinsManagementClient
msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
Current cloud config:
AzureCloud
urllib3.connectionpool : Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool : http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 1958
msrestazure.azure_active_directory : MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
urllib3.connectionpool : Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool : http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 1958
msrestazure.azure_active_directory : MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
msrest.http_logger : Request URL: 'https://management.azure.com/subscriptions/8ec7bf83-77ce-43b0-905f-69753eb35f96/resourceGroups/{REDACTED}/providers/Microsoft.DigitalTwins/digitalTwinsInstances/{REDACTED}?api-version=2020-03-01-preview'
msrest.http_logger : Request method: 'GET'
msrest.http_logger : Request headers:
msrest.http_logger :     'Accept': 'application/json'
msrest.http_logger :     'accept-language': 'en-US'
msrest.http_logger :     'User-Agent': 'python/3.6.10 (Linux-4.15.0-1092-azure-x86_64-with-debian-stretch-sid) msrest/0.6.9 cloud-shell/1.0 msrest_azure/0.6.3 azuredigitaltwinsmanagementclient/2020-03-01-preview IoTPlatformCliExtension/0.9.7 AZURECLI/2.9.1 (DEB)'
msrest.http_logger : Request body:
msrest.http_logger : None
msrest.universal_http : Configuring redirects: allow=True, max=30
msrest.universal_http : Configuring request: timeout=100, verify=True, cert=None
msrest.universal_http : Configuring proxies: ''
msrest.universal_http : Evaluate proxies against ENV settings: True
urllib3.connectionpool : Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool : https://management.azure.com:443 "GET /subscriptions/{REDACTED}/resourceGroups/{REDACTED}/providers/Microsoft.DigitalTwins/digitalTwinsInstances/{REDACTED}?api-version=2020-03-01-preview HTTP/1.1" 200 None
msrest.http_logger : Response status: 200
msrest.http_logger : Response headers:
msrest.http_logger :     'Cache-Control': 'no-cache'
msrest.http_logger :     'Pragma': 'no-cache'
msrest.http_logger :     'Transfer-Encoding': 'chunked'
msrest.http_logger :     'Content-Type': 'application/json; charset=utf-8'
msrest.http_logger :     'Content-Encoding': 'gzip'
msrest.http_logger :     'Expires': '-1'
msrest.http_logger :     'Vary': 'Accept-Encoding'
msrest.http_logger :     'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
msrest.http_logger :     'Server': 'Kestrel'
msrest.http_logger :     'x-ms-ratelimit-remaining-subscription-reads': '11987'
msrest.http_logger :     'x-ms-request-id': 'fe7dcefe-57c1-4c70-ab56-61a42b7580bf'
msrest.http_logger :     'x-ms-correlation-request-id': 'fe7dcefe-57c1-4c70-ab56-61a42b7580bf'
msrest.http_logger :     'x-ms-routing-request-id': 'WESTEUROPE:20200811T130301Z:fe7dcefe-57c1-4c70-ab56-61a42b7580bf'
msrest.http_logger :     'X-Content-Type-Options': 'nosniff'
msrest.http_logger :     'Date': 'Tue, 11 Aug 2020 13:03:01 GMT'
msrest.http_logger : Response content:
msrest.http_logger : {"id":"/subscriptions/{REDACTED}/resourceGroups/digital-twin-rg-20200629/providers/Microsoft.DigitalTwins/digitalTwinsInstances/{REDACTED}","name":"{REDACTED}","type":"Microsoft.DigitalTwins/digitalTwinsInstances","properties":{"provisioningState":"Succeeded","createdTime":"2020-06-29T12:11:19.478833Z","lastUpdatedTime":"2020-06-29T12:11:30.40905Z","hostName":"{REDACTED}.api.neu.digitaltwins.azure.net"},"location":"northeurope","sku":{"name":"S1"},"tags":null}
msrest.universal_http.requests : Configuring retry: max_retries=4, backoff_factor=0.8, max_backoff=90
msrest.async_paging : Paging async iterator protocol is not available for EventRoutePaged
urllib3.connectionpool : Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool : http://localhost:50342 "POST /oauth2/token HTTP/1.1" 400 581
msrestazure.azure_active_directory : MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://digitaltwins.azure.net'}
msrestazure.azure_active_directory : MSI: Failed to retrieve a token from 'http://localhost:50342/oauth2/token' with an error of '400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token'. This could be caused by the MSI extension not yet fully provisioned.
cli.azure.cli.core.util : The command failed with an unexpected error. Here is the traceback:

The command failed with an unexpected error. Here is the traceback:

cli.azure.cli.core.util : 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token Traceback (most recent call last): File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke cmd_result = self.invocation.execute(args) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 654, in execute raise ex File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 718, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 711, in _run_job six.reraise(sys.exc_info()) File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise raise value File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 701, in _run_job result = list(result) File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 143, in next self.advance_page() File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 129, in advance_page self._response = self._get_next(self.next_link) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/sdk/digitaltwins/operations/event_routes_operations.py", line 93, in internal_paging response = self._client.send(request, stream=False, operation_config) File "/opt/az/lib/python3.6/site-packages/msrest/service_client.py", line 336, in send pipeline_response = self.config.pipeline.run(request, kwargs) File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/init.py", line 197, in run return first_node.send(pipeline_request, kwargs) # type: ignore File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/init.py", line 150, in send response = self.next.send(request, kwargs) File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/requests.py", line 65, in send self._creds.signed_session(session) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 31, in signed_session return self.refresh_session(session) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 47, in refresh_session session.headers["Authorization"] = self.generate_token() File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 54, in generate_token creds, subscription, tenant = profile.get_raw_token(resource=self.resource_id) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 643, in get_raw_token creds = self._get_token_from_cloud_shell(resource) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 391, in _get_token_from_cloud_shell auth = MSIAuthenticationWrapper(resource=resource) File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 592, in init self.set_token() File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 598, in settoken self.scheme, , self.token = get_msi_token(self.resource, self.port, self.msi_conf) File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 486, in get_msi_token result.raise_for_status() File "/opt/az/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token Traceback (most recent call last): File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke cmd_result = self.invocation.execute(args) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 654, in execute raise ex File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 718, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 711, in _run_job six.reraise(sys.exc_info()) File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise raise value File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 701, in _run_job result = list(result) File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 143, in next self.advance_page() File "/opt/az/lib/python3.6/site-packages/msrest/paging.py", line 129, in advance_page self._response = self._get_next(self.next_link) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/sdk/digitaltwins/operations/event_routes_operations.py", line 93, in internal_paging response = self._client.send(request, stream=False, operation_config) File "/opt/az/lib/python3.6/site-packages/msrest/service_client.py", line 336, in send pipeline_response = self.config.pipeline.run(request, kwargs) File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/init.py", line 197, in run return first_node.send(pipeline_request, kwargs) # type: ignore File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/init.py", line 150, in send response = self.next.send(request, kwargs) File "/opt/az/lib/python3.6/site-packages/msrest/pipeline/requests.py", line 65, in send self._creds.signed_session(session) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 31, in signed_session return self.refresh_session(session) File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 47, in refresh_session session.headers["Authorization"] = self.generate_token() File "/home/{REDACTED}/.azure/cliextensions/azure-iot/azext_iot/digitaltwins/providers/auth.py", line 54, in generate_token creds, subscription, tenant = profile.get_raw_token(resource=self.resource_id) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 643, in get_raw_token creds = self._get_token_from_cloud_shell(resource) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/_profile.py", line 391, in _get_token_from_cloud_shell auth = MSIAuthenticationWrapper(resource=resource) File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 592, in init self.set_token() File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 598, in settoken self.scheme, , self.token = get_msi_token(self.resource, self.port, self.msi_conf) File "/opt/az/lib/python3.6/site-packages/msrestazure/azure_active_directory.py", line 486, in get_msi_token result.raise_for_status() File "/opt/az/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:50342/oauth2/token cli.azure.cli.core.util : To open an issue, please run: 'az feedback'

To open an issue, please run: 'az feedback' Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f159cf1ee18>] az_command_data_logger : exit code: 1 telemetry.save : Save telemetry record of length 5437 in cache telemetry.check : Negative: The /home/{REDACTED}/.azure/telemetry.txt was modified at 2020-08-11 13:01:55.345532, which in less than 600.000000 s Command ran in 0.543 seconds (init: 0.045, invoke: 0.498)


## Expected Behavior
The command should show a list of created routes.

## Environment Summary

Linux-4.15.0-1092-azure-x86_64-with-debian-stretch-sid (Cloud Shell) Python 3.6.10 Installer: DEB

azure-cli 2.9.1 *

Extensions: azure-iot 0.9.7


## Additional Context

<!--Please don't remove this:-->
<!--auto-generated-->

digimaun commented 4 years ago

Hi @MatthijsvdVeer , thanks for submitting this issue. There is an ongoing problem in CloudShell, the workaround is to run 'az login' prior to running the command. That should resolve your problem. Please give it a try.

MatthijsvdVeer commented 4 years ago

Hi @digimaun, I tried your workaround and it works. Thanks for that. Is there a fix in the making, and will you leave this issue open until that's released? I just noticed another similar issue that's closed.

digimaun commented 4 years ago

There is a highly referenced issue thread we have been linking to bring more visibility here https://github.com/Azure/azure-cli/issues/11749. A fix is in the making to resolve this pain point - but I can't speak to when that will be available. We will leave this open until the fix is released.

digimaun commented 4 years ago

I'm happy to follow up that the cloudshell issue should be fixed for digital twins. You should no longer need to az login prior to running the data plane az dt [model, twin, route] commands in cloudshell.

Closing this issue. If you still run into issues, we can re-open.

Azure / azure-iot-cli-extension

az dt route list Crashes on execution #231

Describe the bug

To Reproduce: