Can't read incoming messages in pytests

[Waganama]

Hi, i'm using aiomqtt to run some test with pytests on Windows.

I've managed to create, connect to my client and also subscribe publish to some topics. The problem is I can't read any messages from my client. The loop 'async for' get stuck.

async with Client(hostname=broker_adress, port = broker_port, tls_context=tls_context, logger=logger) as mqtt_client:
  for device in devices:
    await mqtt_client.subscribe(f"mqtt/things/{device}/uplink")
  topic = f"mqtt/things/{device}/downlink"
  message = "some message"
  await mqtt_client.publish(topic=topic, payload=message)

  async for message in mqtt_client.messages: # Stuck here
    print(message.payload) # Never called

I make sure that my asyncio loop is a SelectorEventLoop, I mark my async test function with @pytest.mark.asyncio I know that i received some messages, the function _messages from client class call yield task.result().

DEBUG    aiomqtt:client.py:3254 Sending PUBLISH (d0, q0, r0, m7), 'b'mqtt/things/E41A1D0000000366/downlink'', ... (268 bytes)
DEBUG    aiomqtt:client.py:473 _messages yield task.result()
DEBUG    aiomqtt:client.py:473 _messages yield task.result()
DEBUG    aiomqtt:client.py:473 _messages yield task.result()

I use python 3.11.4, asyncio-0.21.0 , aiomqtt 2.1.0, pytest 7.3.2 and paho-mqtt 2.0.0

Any suggestion why, I have this behaviour ?

[frederikaalund]

Hi Waganama, thanks for opening this issue. :+1: Let me have a look. :)

It looks like you subscribe to these topics:

f"mqtt/things/{device}/uplink"

but you post messages to this topic (where device is the last entry in devices):

f"mqtt/things/{device}/downlink"

Since the topics don't match, you'll never receive a message. That explains why it gets "stuck": There are no messages in wait.

As a test, try to subscribe to everything: "mqtt/things/#". See if that works. :)

[Waganama]

Thank you for your response. I have two topics, one to send downlink (client->server) and one to received uplink (server->client) I still tried to change the topic for my subscribtion, without success.

It looks like that calling yield task.result() doesn't do anything when running with pytest. I'm also sure that i received message according to these logs i had:

DEBUG    aiomqtt:client.py:618 _on_message _queue size after put 1
DEBUG    aiomqtt:client.py:474 _messages yield task.result()
DEBUG    aiomqtt:client.py:463 _messages _queue size after get 0

I tried to run the same code above without pytest, and it worked perfectly, I can read my message.

[frederikaalund]

Thanks for the response. This is really difficult to debug without the full code. Do you have self-sufficient example code that reproduces the problem?

[Waganama]

I reduce the test part to reproduce the issue only. It is a very simple test. Unfortunatly I can't share connection information and i don't know if there are mqtt server dedicated for test. I hope will have everything to debug.

[Waganama]

After research, i found out i can use the server test.mosquitto.org for my test. I also have the same issue withe the following code using pytest.

Output :

configfile: pytest.ini
plugins: anyio-3.7.0, asyncio-0.21.0
asyncio: mode=Mode.AUTO
collected 2 items

tests/test_mosquitto.py::TestMosquitto::test_mosquitto[asyncio]
--------------------------------------------------- live log setup ----------------------------------------------------
DEBUG    asyncio:selector_events.py:54 Using selector: SelectSelector
---------------------------------------------------- live log call ----------------------------------------------------
INFO     root:test_mosquitto.py:37 Here
DEBUG    aiomqtt:client.py:3254 Sending CONNECT (u0, p0, wr0, wq0, wf0, c1, k60) client_id=b''
DEBUG    aiomqtt:client.py:3254 Received CONNACK (0, 0)
DEBUG    aiomqtt:client.py:3254 Sending SUBSCRIBE (d0, m1) [(b'win32_3.11/tests/aiomqtt/test_client_unsubscribe/1', 0)]
DEBUG    aiomqtt:client.py:3254 Received SUBACK
DEBUG    aiomqtt:client.py:3254 Sending PUBLISH (d0, q0, r0, m2), 'b'win32_3.11/tests/aiomqtt/test_client_unsubscribe/1'', ... (12 bytes)
DEBUG    aiomqtt:client.py:463 _messages _queue size after get 0
DEBUG    aiomqtt:client.py:3254 Received PUBLISH (d0, q0, r0, m0), 'win32_3.11/tests/aiomqtt/test_client_unsubscribe/1', ...  (12 bytes)
DEBUG    aiomqtt:client.py:618 _on_message _queue size after put 1
DEBUG    aiomqtt:client.py:474 _messages yield task.result().topic : win32_3.11/tests/aiomqtt/test_client_unsubscribe/1
DEBUG    aiomqtt:client.py:463 _messages _queue size after get 0

source.zip

[Waganama]

I have additionnal informations, if i have different behaviour according how I run my test :

• python -m pytest tests\test_mosquitto.py -> No message is printed after the async for loop
• python -m pytest --capture=no tests\test_mosquitto.py -> Message are printed

I don't know why the capture=no has effect.

[Waganama]

It turned out that everything was fine, pytest doesn't print by default.

I'm sorry for the inconvenience

[frederikaalund]

No problem. 😄 Glad that it worked out in the end!