MkKiefer commented 1 year ago

Amqp over websocket connection error

UAMQP Package Version: 1.6.3
Operating System: Windows / WSL / Docker
Python Version: 3.8, 3.9, 3.10

Describe the bug:

Connection to IoTHub failed with transport type: amqp.TransportType.AmqpOverWebsocket at versions > v1.2.13
Currently using the version: v1.2.13 (the last version where the amqp over ws works for me) and trying to update to v1.6.3 The websocket connection is needed because some firewalls are blocking the plain amqp connection This behavior is the same on multiple platforms:

x86_x64
Armv7 (Raspberry 3b+)
Armv8 (Raspberry 4)

The same error ocurred with the async and sync client.

Logg message

### Logs ```PYTHON 2023-01-16 10:11:32,134 uamqp.c_uamqp DEBUG Creating SASL Mechanism 2023-01-16 10:11:32,138 uamqp.client DEBUG Opening client connection. 2023-01-16 10:11:32,138 uamqp DEBUG Initializing platform. 2023-01-16 10:11:32,138 uamqp.connection INFO Connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' state changed from to 2023-01-16 10:11:32,140 uamqp.c_uamqp DEBUG Wrapping value type: 2023-01-16 10:11:32,141 uamqp.c_uamqp DEBUG Wrapping value type: 2023-01-16 10:11:32,142 uamqp.sender INFO Message sender b'sender-link-2aaa5d30-aa2d-4a79-a028-ee3168794609' state changed from to on connection: b'7475125f-f01a-4a3a-936c-7fdc8c69f607' 2023-01-16 10:11:32,224 uamqp.connection INFO Connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' state changed from to 2023-01-16 10:11:32,553 uamqp.c_uamqp INFO b'wsio_close when not open.' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\deps\\azure-c-shared-utility\\src\\wsio.c':b'internal_close':153) 2023-01-16 10:11:32,554 uamqp.connection INFO Connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' state changed from to 2023-01-16 10:11:32,554 uamqp.connection INFO Connection with ID b'7475125f-f01a-4a3a-936c-7fdc8c69f607' unexpectedly in an error state. Closing: False, Error: None 2023-01-16 10:11:32,555 uamqp.sender INFO Sender link failed to open - expecting to receive DETACH frame. 2023-01-16 10:11:32,555 uamqp.connection WARNING ConnectionClose('ErrorCodes.UnknownError: Connection in an unexpected error state.') 2023-01-16 10:11:32,556 uamqp.c_uamqp DEBUG Destroying cMessageSender 2023-01-16 10:11:32,556 uamqp.c_uamqp DEBUG Destroying cLink 2023-01-16 10:11:32,557 uamqp.c_uamqp DEBUG Deallocating 'CompositeValue' 2023-01-16 10:11:32,557 uamqp.c_uamqp DEBUG Destroying 'CompositeValue' 2023-01-16 10:11:32,557 uamqp.c_uamqp DEBUG Deallocating 'CompositeValue' 2023-01-16 10:11:32,558 uamqp.c_uamqp DEBUG Destroying 'CompositeValue' 2023-01-16 10:11:32,558 uamqp.c_uamqp DEBUG Deallocating cLink 2023-01-16 10:11:32,558 uamqp.c_uamqp DEBUG Deallocating cMessageSender 2023-01-16 10:11:32,559 uamqp.client DEBUG Closing non-CBS session. 2023-01-16 10:11:32,559 uamqp.c_uamqp DEBUG Destroying cSession 2023-01-16 10:11:32,560 uamqp.c_uamqp DEBUG Deallocating cSession 2023-01-16 10:11:32,560 uamqp.client DEBUG Closing exclusive connection. 2023-01-16 10:11:32,560 uamqp.connection DEBUG Unlocked connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' to close. 2023-01-16 10:11:32,561 uamqp.connection INFO Shutting down connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607'. 2023-01-16 10:11:32,561 uamqp.c_uamqp DEBUG Destroying Connection 2023-01-16 10:11:32,561 uamqp.c_uamqp INFO b'send called while not open' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_send_async':1181) 2023-01-16 10:11:32,562 uamqp.c_uamqp INFO b'Cannot send encoded bytes' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':268) 2023-01-16 10:11:32,562 uamqp.c_uamqp INFO b'saslclientio_close called while not open' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 2023-01-16 10:11:32,562 uamqp.c_uamqp INFO b'xio_close failed' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'on_bytes_encoded':272) 2023-01-16 10:11:32,563 uamqp.connection INFO Connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' state changed from to 2023-01-16 10:11:32,563 uamqp.c_uamqp INFO b'-> [CLOSE]* {}' 2023-01-16 10:11:32,563 uamqp.connection INFO Connection b'7475125f-f01a-4a3a-936c-7fdc8c69f607' state changed from to 2023-01-16 10:11:32,563 uamqp.c_uamqp INFO b'saslclientio_close called while not open' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\saslclientio.c':b'saslclientio_close_async':1130) 2023-01-16 10:11:32,564 uamqp.c_uamqp INFO b'xio_close failed' (b'D:\\a\\1\\s\\src\\vendor\\azure-uamqp-c\\src\\connection.c':b'connection_close':1437) 2023-01-16 10:11:32,564 uamqp.c_uamqp DEBUG Destroying XIO 2023-01-16 10:11:32,564 uamqp.c_uamqp DEBUG Destroying XIO 2023-01-16 10:11:32,565 uamqp.c_uamqp DEBUG Destroying SASLMechanism 2023-01-16 10:11:32,565 uamqp.connection INFO Connection shutdown complete b'7475125f-f01a-4a3a-936c-7fdc8c69f607'. 2023-01-16 10:11:32,565 uamqp DEBUG Deinitializing platform. ```

System / dependency's

WSL

Test inside docker container with different build environments:

Dockerfile: python 3.8
Dockerfile: python 3.9

RaspberryPi 4 / RaspberryPi3b+

Test inside docker container with different build environments:

Dockerfile: python 3.8
Dockerfile: python 3.9

The test where also made using the install openssl install script
(https://github.com/Azure/azure-uamqp-python/blob/main/utils/install_openssl.sh)
FROM python:3.8
RUN apt-get update
RUN apt-get install -y build-essential libssl-dev uuid-dev cargo curl cmake libcurl4-openssl-dev pkg-config
RUN curl https://sh.rustup.rs -sSf | bash -s -- -y
RUN pip install -U pip \
&& pip install -r requirements.txt \
--no-binary :all:

Windows:

Python on Windows:

python 3.8
python 3.9
python 3.10

pip

pip list (venv)

certifi    2022.12.7
pip        22.3.1   
setuptools 56.0.0   
six        1.16.0   
uamqp      1.6.3

To Reproduce

Create a IoTHub instance (currently using Baltimore CyberTrust root but also tested with DigiCert Global G2)
Add a device and get the connection string
Try to connect and send a message with uamqp.SendClient(uri, debug=True, transport_type=uamqp.TransportType.AmqpOverWebsocket) (see example)

Example code
### Send to IoTHub ```PYTHON logging.basicConfig(level=logging.DEBUG, format='%(asctime)s %(name)s %(levelname)s >> %(message)s', datefmt='%Y-%m-%d %H:%M:%S')

def generate_connection_uri(config): """ Generates the amqp enpoint url depending on the provided connection string :return: amqp endpoint """ username = '{device_id}@sas.{iot_hub_name}'.format( device_id=config["device_id"], iot_hub_name=config["hub_name"])

# Create token
ttl = int(time.time()) + 3600
uri = '{hostname}/devices/{device_id}'.format(
        hostname=config["hub_address"],
        device_id=config["device_id"])
url_to_sign = urllib.parse.quote(uri, safe='')
hash_auth = hmac.new(
    base64.b64decode(config["device_key"]),
    msg="{0}\n{1}".format(url_to_sign, ttl).encode('utf-8'),
    digestmod='sha256'
)
sas_token = "SharedAccessSignature sr={0}&sig={1}&se={2}".format(
    url_to_sign,
    urllib.parse.quote(base64.b64encode(hash_auth.digest()), safe=''),
    ttl
)

operation = '/devices/{device_id}/messages/events'.format(device_id=config["device_id"])
uri = 'amqps://{}:{}@{}{}'.format(
    quote_plus(username),
    quote_plus(sas_token),
    config["hub_address"],
    operation)
return uri

def send_iot_hub(config): data = b"test message" message = uamqp.Message(data) uri = generate_connection_uri(config)

client = uamqp.SendClient(uri, debug=True, transport_type=uamqp.TransportType.AmqpOverWebsocket)
client.queue_message(message)
states = client.send_all_messages()
assert not [state for state in states if state == MessageState.SendFailed]

if name == 'main': config = {} REG_EXP_CONNECTION_STRING = r"^HostName=(.).azure-devices.net;DeviceId=(.);SharedAccessKey=(.*)$" m_obj = re.match(REG_EXP_CONNECTION_STRING, os.environ['CONNECTION_STRING']) config['hub_address'] = m_obj.group(1) + ".azure-devices.net" config['hub_name'] = m_obj.group(1) config['device_id'] = m_obj.group(2) config['device_key'] = m_obj.group(3) print(config)

send_iot_hub(config)


</details>

## Expected behavior
Expected would be a amqp connection over websocket to the IoTHub like with plain amqp (no connection errors)

MkKiefer commented 1 year ago

It looks like the default port for amqp over websocket is not handled correctly. It is possible to set the port in the auth (SASLPlain) object directly. That is the workaround that i now use:

    auth = uamqp.authentication.SASLPlain(
        hostname="<hostname>",
        username="<username>",
        password="<password>",
        port=443,
        transport_type=uamqp.TransportType.AmqpOverWebsocket
    )
    client = uamqp.SendClient(uri, auth, debug=True, transport_type=uamqp.TransportType.AmqpOverWebsocket)

kashifkhan commented 1 year ago

Closing this issue as the PR is merged. Thank you @MkKiefer

Azure / azure-uamqp-python

Amqp over websocket is not working and leeds to a connection error #355