search

Azure / azure-iot-cli-extension

Azure IoT extension for Azure CLI
Other
82 stars 65 forks source link

az iot hub monitor --login fails #597

Closed rajkan01 closed 1 year ago

rajkan01 commented 1 year ago

Describe the bug

az iot hub monitor --login throws on exception at azext_iot monitor python package

CLI version

az version
{
  "azure-cli": "2.42.0",
  "azure-cli-core": "2.42.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "azure-iot": "0.18.1"
  }
}

To Reproduce Steps to reproduce the behavior:

  1. Run the command az iot hub monitor-events --hub-name device --device-id 12345 --login 'HostName=***********;SharedAccessKeyName=***********;SharedAccessKey=**********'
  2. See error

Expected behavior Monitor device telemetry & messages sent to an IoT Hub.

Screenshots

The command failed with an unexpected error. Here is the traceback:
'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/use/.venv/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/use/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/home/use/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/use/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/home/use/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/home/use/.venv/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2421, in iot_hub_monitor_events
    _iot_hub_monitor_events(
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2542, in _iot_hub_monitor_events
    target = hub_target_builder.EventTargetBuilder().build_iot_hub_target(target)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 38, in build_iot_hub_target
    return self.eventLoop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 88, in _build_iot_hub_target_async
    partition_count = meta_data[b"partition_count"]
TypeError: 'NoneType' object is not subscriptable
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
To open a new issue, please run `az feedback`
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Environment (please complete the following information):

Additional context Add any other context about the problem here.

c-ryan-k commented 1 year ago

Hey @rajkan01 - I setup an ubuntu 22 environment to mimic yours and I'm not able to reproduce the issue.

A couple of things to check / try:

rajkan01 commented 1 year ago

Hi @c-ryan-k Please find my inline answers

Adding the -r switch to this command will force a repair of uamqp 1.2 before running the command [RA] I tried that but didn't help

Do other commands work with --login and this connection string? [RA]Could you share the other command which supports --login

Does this command work without using --login auth? [RA] Yes it works but I need to do login manually with my cloud credentials

Does your hub have any features enabled? You can check this with az iot hub show -n [hub-name] --query "properties.features" [RA] I got the response "None"

But still, I have a problem with my setup

c-ryan-k commented 1 year ago

Ok, appreciate the quick response.

This issue seems to stem from an empty response (or possibly an exception on connection) to the amqp endpoint using the auth values provided in the connection string. Off the top of my head, it's possible that either this connection string / key doesn't have the correct scope, or that the hub is refusing / redirecting the amqp endpoint connection for some reason.

Let's see if we can validate your connection string and policy key before we take a look at the hub side of things.

Is this a custom created/scoped connection string, or is this for a default policy like iothubowner?

Just to make sure, can you see which rights your policy for the connection string has by using the command: az iot hub policy show --hub-name [hub_name] --name [sharedAccessKeyName] --query "rights"

You can try any other dataplane command (device-identity commands, as an example) with --login and without referencing the hub name like so:

az iot hub device-identity list --login "connection_string"

One more thing to try - if the connection string works ok - run the same monitor command with the --debug switch and watch for uamqp.* debug lines:

az iot hub monitor-events --login "connection_string"

You should see a bit of uamqp.connection and uamqp.authentication log chatter before the error occurs, perhaps there's some data points in there that could help troubleshoot (auth failures, connection rejected, etc)

rajkan01 commented 1 year ago

Please see Cmd: az iot hub policy show --hub-name [hub_name] --name [sharedAccessKeyName] --query "rights" Result: 

Error loading command module 'iot': MGMT_IOT
'policy' is misspelled or not recognized by the system.

Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs

Cmd: az iot hub device-identity list --login "connection_string" Result:

[
  {
    "authenticationType": "certificateAuthority",
    "capabilities": {
      "iotEdge": false
    },
    "cloudToDeviceMessageCount": 0,
    "connectionState": "Disconnected",
    "deviceEtag": "MzAyMjYxNjA3",
    "deviceId": "215383a3-5d10-400a-9692-0c8db49c4886",
    "etag": "AAAAAAAAAAE=",
    "lastActivityTime": "2022-11-22T15:14:05.9856083Z",
    "modelId": "",
    "properties": {
      "desired": {
        "$metadata": {
          "$lastUpdated": "2022-10-22T21:05:04.4813564Z"
        },
        "$version": 1
      },
      "reported": {
        "$metadata": {
          "$lastUpdated": "2022-10-22T21:05:04.4813564Z"
        },
        "$version": 1
      }
    },
    "status": "enabled",
    "statusUpdateTime": "0001-01-01T00:00:00Z",
    "version": 2,
    "x509Thumbprint": {
      "primaryThumbprint": null,
      "secondaryThumbprint": null
    }
  }
]

Cmd: az iot hub monitor-events --login "connection_string" --debug Result:

cli.knack.cli: Command arguments: ['iot', 'hub', 'monitor-events', '--login', 'HostName=device.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=1234444', '--debug']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f1a981dec20>, <function OutputProducer.on_global_arguments at 0x7f1a982704c0>, <function CLIQuery.on_global_arguments at 0x7f1a98089900>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'iot': ['azext_iot']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: Total (0)                 0.000         0         0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name                  Load Time    Groups  Commands  Directory
cli.azure.cli.core: azure-iot                 0.021        92       302  /home/user/.azure/cliextensions/azure-iot
cli.azure.cli.core: Total (1)                 0.021        92       302  
cli.azure.cli.core: Loaded 92 groups, 302 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command  : iot hub monitor-events
cli.azure.cli.core: Command table: iot hub monitor-events
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f1a9765f910>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/user/.azure/commands/2022-11-28.10-34-53.iot_hub_monitor-events.79875.log'.
az_command_data_logger: command args: iot hub monitor-events --login {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7f1a97684700>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7f1a976ab0a0>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7f1a976ab1c0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f1a98270550>, <function CLIQuery.handle_query_parameter at 0x7f1a98089990>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7f1a976ab130>]
az_command_data_logger: extension name: azure-iot
az_command_data_logger: extension version: 0.18.1
asyncio: Using selector: EpollSelector
uamqp.c_uamqp: Creating SASL Mechanism
uamqp: Initializing platform.
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
uamqp.c_uamqp: Wrapping value type: <AMQPType.CompositeType: 23>
uamqp.c_uamqp: Wrapping value type: <AMQPType.CompositeType: 23>
uamqp.receiver: Message receiver b'receiver-link-2b3fc51c-78d0-4526-830c-18451e647683' state changed from <MessageReceiverState.Idle: 1> to <MessageReceiverState.Opening: 2> on connection: b'e64936bc-a11b-41da-9968-692bd7b0b236'
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
uamqp.c_uamqp: Link ATTACH frame missing source and/or target. DETACH pending.
uamqp.receiver: Receiver link ATTACH frame invalid - expecting to receive DETACH frame.
uamqp.c_uamqp: Wrapping value type: <AMQPType.DictValue: 20>
uamqp.c_uamqp: Wrapping value type: <AMQPType.SymbolValue: 18>
uamqp.c_uamqp: Wrapping value type: <AMQPType.StringValue: 17>
uamqp.c_uamqp: Wrapping value type: <AMQPType.SymbolValue: 18>
uamqp.c_uamqp: Wrapping value type: <AMQPType.StringValue: 17>
uamqp.c_uamqp: Deallocating 'SymbolValue'
uamqp.c_uamqp: Destroying 'SymbolValue'
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Wrapping value type: <AMQPType.SymbolValue: 18>
uamqp.c_uamqp: Wrapping value type: <AMQPType.IntValue: 9>
uamqp.c_uamqp: Deallocating 'SymbolValue'
uamqp.c_uamqp: Destroying 'SymbolValue'
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Wrapping value type: <AMQPType.SymbolValue: 18>
uamqp.c_uamqp: Wrapping value type: <AMQPType.StringValue: 17>
uamqp.c_uamqp: Deallocating 'SymbolValue'
uamqp.c_uamqp: Destroying 'SymbolValue'
uamqp.c_uamqp: Deallocating 'IntValue'
uamqp.c_uamqp: Destroying 'IntValue'
uamqp.c_uamqp: Deallocating 'SymbolValue'
uamqp.c_uamqp: Destroying 'SymbolValue'
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Deallocating 'DictValue'
uamqp.c_uamqp: Destroying 'DictValue'
uamqp.receiver: Received Link detach event: b'amqp:link:redirect'
Link: b'receiver-link-2b3fc51c-78d0-4526-830c-18451e647683'
Description: None
Details: {b'hostname': b'iothub-ns-secure-inf-21885297-72e06a9710.servicebus.windows.net', b'network-host': b'iothub-ns-secure-inf-21885297-72e06a9710.servicebus.windows.net', b'port': 5671, b'address': b'amqps://iothub-ns-secure-inf-21885297-72e06a9710.servicebus.windows.net:5671/secure-inference-device/$management'}
Retryable: False
Connection: b'e64936bc-a11b-41da-9968-692bd7b0b236'
uamqp.c_uamqp: Deallocating cError
uamqp.receiver: Receiver link ATTACH frame invalid - expecting to receive DETACH frame.
uamqp.receiver: LinkRedirect('ErrorCodes.LinkRedirect')
uamqp.c_uamqp: Destroying cMessageReceiver
uamqp.c_uamqp: Destroying cLink
uamqp.async_ops.client_async: Closing non-CBS session.
uamqp.c_uamqp: Destroying cSession
uamqp.async_ops.client_async: Closing exclusive connection b'e64936bc-a11b-41da-9968-692bd7b0b236'.
uamqp.async_ops.connection_async: Unlocked connection b'e64936bc-a11b-41da-9968-692bd7b0b236' to close.
uamqp.async_ops.connection_async: Shutting down connection b'e64936bc-a11b-41da-9968-692bd7b0b236'.
uamqp.c_uamqp: Destroying Connection
uamqp.connection: Connection b'e64936bc-a11b-41da-9968-692bd7b0b236' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.END: 13>
uamqp.c_uamqp: Destroying XIO
uamqp.c_uamqp: Destroying XIO
uamqp.c_uamqp: Destroying SASLMechanism
uamqp.async_ops.connection_async: Connection shutdown complete b'e64936bc-a11b-41da-9968-692bd7b0b236'.
uamqp: Deinitializing platform.
uamqp: Platform already initialized.
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.START: 0> to <ConnectionState.START: 0>
uamqp.c_uamqp: Deallocating 'AMQPValue'
uamqp.c_uamqp: Destroying 'AMQPValue'
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.START: 0> to <ConnectionState.HDR_SENT: 2>
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.HDR_SENT: 2> to <ConnectionState.HDR_EXCH: 3>
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.HDR_EXCH: 3> to <ConnectionState.OPEN_SENT: 7>
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.OPEN_SENT: 7> to <ConnectionState.OPENED: 9>
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating cTarget
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating cTarget
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: CBS for connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' completed opening with status: 1
uamqp.c_uamqp: Token put complete with result: 1, status: 202, description: b'Accepted', connection: b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating cTarget
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating cTarget
uamqp.c_uamqp: Destroying cTarget
uamqp.c_uamqp: Management link open: 1
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Deallocating 'DictValue'
uamqp.c_uamqp: Destroying 'DictValue'
uamqp.c_uamqp: Management op complete: 3, status code: 401, description: b'Attempted to perform an unauthorized operation.'
uamqp.async_ops.mgmt_operation_async: Failed to complete mgmt operation.
Status code: 401
Message: b'Attempted to perform an unauthorized operation.'
uamqp.message: Parsing received message None.
uamqp.c_uamqp: Wrapping value type: <AMQPType.BinaryValue: 16>
uamqp.c_uamqp: Deallocating 'BinaryValue'
uamqp.c_uamqp: Destroying 'BinaryValue'
uamqp.async_ops.client_async: CBS session pending b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.async_ops.client_async: Closing exclusive connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.async_ops.connection_async: Unlocked connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' to close.
uamqp.async_ops.connection_async: Shutting down connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.authentication.cbs_auth_async: Shutting down CBS session on connection: b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.c_uamqp: Destroying CBSTokenAuth for connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'
uamqp.authentication.cbs_auth_async: Auth closed, destroying session on connection: b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.c_uamqp: Destroying cManagementOperation
uamqp.c_uamqp: Destroying cSession
uamqp.authentication.cbs_auth_async: Finished shutting down CBS session on connection: b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp.c_uamqp: Destroying Connection
uamqp.connection: Connection b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a' state changed from <ConnectionState.OPENED: 9> to <ConnectionState.END: 13>
uamqp.c_uamqp: Destroying XIO
uamqp.c_uamqp: Destroying XIO
uamqp.c_uamqp: Destroying SASLMechanism
uamqp.async_ops.connection_async: Connection shutdown complete b'771f1a0b-aec1-43e2-98d7-8bad8061ab8a'.
uamqp: Deinitializing platform.
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Deallocating cMessage
uamqp.c_uamqp: Destroying cMessage
uamqp.c_uamqp: Deallocating cMessage
uamqp.c_uamqp: Destroying cMessage
None
uamqp.c_uamqp: Deallocating cSession
uamqp.c_uamqp: Deallocating 'CompositeValue'
uamqp.c_uamqp: Destroying 'CompositeValue'
uamqp.c_uamqp: Deallocating 'CompositeValue'
uamqp.c_uamqp: Destroying 'CompositeValue'
uamqp.c_uamqp: Deallocating Connection
uamqp.c_uamqp: Deallocating cLink
uamqp.c_uamqp: Deallocating cMessageReceiver
uamqp.c_uamqp: Deallocating cSource
uamqp.c_uamqp: Destroying cSource
uamqp.c_uamqp: Deallocating 'StringValue'
uamqp.c_uamqp: Destroying 'StringValue'
uamqp.c_uamqp: Deallocating SASLMechanism
uamqp.c_uamqp: Deallocating XIO
uamqp.c_uamqp: Deallocating XIO
cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
cli.azure.cli.core.util: Traceback (most recent call last):
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2421, in iot_hub_monitor_events
    _iot_hub_monitor_events(
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2542, in _iot_hub_monitor_events
    target = hub_target_builder.EventTargetBuilder().build_iot_hub_target(target)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 38, in build_iot_hub_target
    return self.eventLoop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 88, in _build_iot_hub_target_async
    partition_count = meta_data[b"partition_count"]
TypeError: 'NoneType' object is not subscriptable

cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
cli.azure.cli.core.azclierror: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2421, in iot_hub_monitor_events
    _iot_hub_monitor_events(
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2542, in _iot_hub_monitor_events
    target = hub_target_builder.EventTargetBuilder().build_iot_hub_target(target)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 38, in build_iot_hub_target
    return self.eventLoop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 88, in _build_iot_hub_target_async
    partition_count = meta_data[b"partition_count"]
TypeError: 'NoneType' object is not subscriptable
az_command_data_logger: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2421, in iot_hub_monitor_events
    _iot_hub_monitor_events(
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/operations/hub.py", line 2542, in _iot_hub_monitor_events
    target = hub_target_builder.EventTargetBuilder().build_iot_hub_target(target)
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 38, in build_iot_hub_target
    return self.eventLoop.run_until_complete(
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/home/user/.azure/cliextensions/azure-iot/azext_iot/monitor/builders/hub_target_builder.py", line 88, in _build_iot_hub_target_async
    partition_count = meta_data[b"partition_count"]
TypeError: 'NoneType' object is not subscriptable
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
To open a new issue, please run `az feedback`
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f1a9765fb50>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 1.838 seconds (init: 0.116, invoke: 1.722)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 6960 in cache
telemetry.check: Returns Positive.
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/home/user/projects/zephyrproject/.venv/bin/python3 /home/user/projects/zephyrproject/.venv/lib/python3.10/site-packages/azure/cli/telemetry/__init__.py /home/user/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:

I noticed the error from the monitor-events cli command logs

uamqp.async_ops.mgmt_operation_async: Failed to complete mgmt operation.
Status code: 401
Message: b'Attempted to perform an unauthorized operation.'

I am not sure what is the problem with my cloud environment, any suggestions?

riccardominato commented 1 year ago

I'm having exactly the same problem. It stopped working overnight with no apparent reason.

{
  "azure-cli": "2.42.0",
  "azure-cli-core": "2.42.0",
  "azure-cli-telemetry": "1.0.8",
  "extensions": {
    "azure-iot": "0.18.1"
  }
}

OS: macOS 12.6 Shell: zsh Python version: 3.10.8

Other colleagues have the same problem using the same key with different configurations, though.

digimaun commented 1 year ago

@rajkan01 , @riccardominato I want to rule out there are no IP filter configurations at play -- @riccardominato by exact same problem, you are also getting a 401 error with the amqp: link: redirect flow calling the $management node of the default/backing event hub...right? And this issue happened without any client updates?

digimaun commented 1 year ago

Something to try as a sanity check which would help build context for this problem is to create a new vanilla IoT Hub in your subscription, run monitor-events with --login and see if there is a failure against the new hub.

digimaun commented 1 year ago

We are actively investigating this on the service side with details provided. For a more targeted investigation I would suggest creating an Azure support ticket.

riccardominato commented 1 year ago

I've just tried again and the error has disappeared. Nothing changed in IotHub configurations.

digimaun commented 1 year ago

@riccardominato thanks for reporting back on this, @rajkan01 does the problem still persist for you?

digimaun commented 1 year ago

The service side net result of investigation classified this problem as a transient issue.

If you still run into this problem, or are not satisfied with the investigation result we recommend opening an Azure support ticket for a more focused investigation of your environment.