[bug] az iot hub monitor-events -n {iothub_name} is not working

[afangster]

Describe the bug Command used:

az iot hub monitor-events -n {iothub_name}

Errors:

Starting event monitor, use ctrl-c to stop...
ErrorCodes.UnknownError: Connection in an unexpected error state.
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:

More details when run az iot hub monitor-events --login {connection string}:

ErrorCodes.UnknownError: Connection in an unexpected error state.
Traceback (most recent call last):
...
azure-iot\azext_iot\operations\hub.py", line 2548, in iot_hub_monitor_events
    _iot_hub_monitor_events(
...
...\cliextensions\azure-iot\uamqp\async_ops\client_async.py", line 947, in receive_message_batch_async
    receiving = await self.do_work_async()
...\cliextensions\azure-iot\uamqp\async_ops\client_async.py", line 374, in do_work_async
    if not await self.client_ready_async():
...\cliextensions\azure-iot\uamqp\async_ops\client_async.py", line 359, in client_ready_async
    await self._connection.work_async()
...\cliextensions\azure-iot\uamqp\async_ops\connection_async.py", line 129, in work_async
    raise self._error
uamqp.errors.ConnectionClose: ErrorCodes.UnknownError: Connection in an unexpected error state.

I have run this command in my Azure Cloud Shell and it works fine there. However, it fails when I run it locally as given above.

To Reproduce Steps to reproduce the behavior:

1. Download and log in to Azure CLI on powershell.
2. Run az iot hub monitor-events -n {iothub_name}
3. See error

Expected behavior I expected to see a live updated list of my sensor data as I did when running it on Azure Cloud Shell. Example as follows:

{
    "event": {
        "origin": "----",
        "module": "",
        "interface": "----",
        "component": "----",
        "payload": "{data here}"
    }
}

Environment (please complete the following information):

• OS: [Windows 10]
• Shell: [powershell]
• Az CLI version: [2.49.0]
• azure-iot extension version: [0.21.3]
• Python (windows): [3.10.10]

Additional context Add any other context about the problem here.

[vilit1]

Can you first try running the az iot hub monitor-events command with the repair flag (--repair). This should reinstall uamqp to the correct version. You can also do pip freeze to check the uamqp version installed.

Also for curiosity, can you let me know what version of uamqp you currently have installed (before running the command with the repair flag).

[afangster]

Running az iot hub monitor-events -n iothub --repair true still resulted in:

Dependency update (uamqp 1.2) required for IoT extension version: 0.21.3.
Continue? (y/n) -> y
Updating required dependency...
Update complete. Executing command...
Starting event monitor, use ctrl-c to stop...
ErrorCodes.UnknownError: Connection in an unexpected error state.
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:
Error in sys.excepthook:

Original exception was:

I am not sure what version I had installed. Running pip freeze yielded no output.

[vilit1]

I have more questions:

1. Are you using --login with the connection string for auth? What auth type did you use for the Azure Cloud Shell?
2. Do other commands for this IoT Hub work (ex: az iot hub device-identity list -n {iothub_name})
3. Does your IoT Hub have any network permissions (ex: IP filter, private endpoint connections)

[vilit1]

You can also do a sanity check by creating a new hub with az iot hub create and running the monitor-events command with that hub.

[afangster]

1. How do I check the auth type? I just followed this thread (https://github.com/Azure/azure-iot-cli-extension/issues/124) when trying to debug with --login. I usually just use -n iothub_name.
2. Other commands do work. Out of all of the commands I have tried, this is the only one that fails.
3. I don't think it does. However, how would I check this? Also, I am not sure if I am allowed to create a new hub...

[vilit1]

1. There are 3 authentication types for the CLI for now: a. --login in which you would provide the connection string for the IoT Hub. We would want this connection string to be one that has the shared access key name as iothubowner. You can get the connection string with az iot hub connection-string show -n {hub_name} --pn iothubowner b. --auth-type key in which the CLI retrieves the necessary connection string for you. c. --auth-type login in which the CLI uses your credentials against RBAC permissions. You can assign defaults to --auth-type, which you can check using az configure -l

2. Do other uamqp dependent commands work for you too (ex: az iot hub monitor-feedback)?

3. The easiest way is to check from the portal, you should go to your hub information and click on networking. If there is any IP filter rules, then Selected IP ranges would be checked (with a list of IP's that the Hub accepts). For Private Endpoints, you need to go to the Private Access tab and check if there are any endpoints. Here are images of my test hub (that has no network configurations enabled that could affect connection).

If this does not work, we recommend opening an Azure support ticket for a more focused investigation of your environment.

[afangster]

1. Running az configure -l on both the cloud shell and on my local powershell just outputs []. I just run az iot hub monitor-events -n iothub on the cloud shell for output. I don't use --login. In my original post, I was running az iot hub monitor-events --login {connection string} with the connection string for iothubowner.
2. az iot hub monitor-feedback works for me. It just says

   Starting C2D feedback monitor, use ctrl-c to stop...

   without any error messages.

3. I do not have any permissions. I am checking with a friend to see if it is a port or network issue on my side. However, where should I file a support ticket?

[vilit1]

1. You will not see any events unless there is an event (ex: you can simulate a device with az iot device simulate)
2. You will not see any feedback unless there is cloud to device messaging (ex: you can simulate a device sending d2c messages with az iot device send-d2c-message)

You can open a support ticket by going to your hub's page in portal and scrolling all the way down on the left menu. There should be a New Support Request link under Support + troubleshooting

[afangster]

Thank you for the help! Honestly have no idea why only this command is not working.

[afangster]

Turns out it was indeed a network issue.

[vilit1]

I will close this issue then. If you see any other issues, feel free to create a new issue within this repo.