Python V5 client unable to see "ResponseTopic" and "CorrelationData" fields

[cornelius-span]

When I publish a message using a V5 client, Rust clients are able to see and extract the "ResponseTopic" and "CorrelationData" fields in the message however a Python client does not register these fields as available. Is this a known issue or is there a specific protocol for sending messages to other language clients?

[fpagliughi]

In order to fully transmit and receive a v5 message intact, the publisher, subscriber, and broker must all support the v5 protocol. But what's more, both the publisher and subscriber clients must connect to the broker using v5.

When the client connects, it must specify the version that it wants to use in the MQTT CONNECT packet. The broker will respond with the accepted version number in the CONNACK packet as a confirmation. Most libraries will give you the informational parts of CONNACK when they alert you that the connection is complete/accepted, like the ServerResponse in the Rust library.

I suspect you're properly connecting with v5 when you're using the Rust library, but maybe connecting with v3.x when using the Python library.

If you're using the Paho Python library you must specify the desired protocol when you create the client object, like:

import paho.mqtt.client as paho

cli = paho.Client(client_id="some-id", protocol=paho.MQTTv5)

I don't know if the MQTT v5 Specification says what should be done in this case, but in practice, if you connect a v3 subscriber to the broker, then any v5 messages that match its subscription topic(s) will be sent to the client - but only with the parts that are included in the v3 spec. So, for example, the payload will be exactly what was sent, but the message will be missing all the properties, since those are not part of the v3 specs.

So if the receiver is supposed to be a v5 RPC service, that would be pretty useless!

Hopefully that answers the question, but if you have any more questions specifically about the Python library, it would be better to ask on the Python repo.

[cornelius-span]

To clarify, I am 100% sure that the Python side is running V5. I am fully able to use said V5 properties when interchanging messages python/python and even C/python. Thanks for the response but I suspect that the python is not the issue. Would you have an example of python<-->paho-rust v5? Because I tried a few different configs on the rust side but no luck.

[cornelius-span]

Nvm believe there was an issue on the Rust side of my code. Sorry for the disturbance.

[fpagliughi]

I use MQTT v5 for Rust <-> Paho Python and/or MQTT.js in production. And I do RPCs using the ResponseTopic and Correlation data, going in both directions across languages. So I know it all works.

But perhaps I didn't understand your original question.

There is absolutely nothing language specific about MQTT, other than how to encode the payloads, which is outside the spec. All the properties are completely defined in type, length, byte order, and all that.

First I guess, check that the properties are on the bus, and if they're making it on both trips... from publisher -> broker and also broker -> subscriber. A tool like WireShark is great for this. Filter on "mqtt" to limit the traffic it shows.

Or just spy on the bus using a v5 capable 3rd party tool to make sure the properties are being sent properly.

But, as for an example... you are in luck! I wrote a simple MQTT v5 RPC client and server for both this repo and the Paho Python client. They can interact in any combination (Rust client calling Python service, Python client calling Rust service, Rust client calling Rust service, etc).

It's probably easiest to clone both repos to your local drive. For the Rust one, just build it with examples, cargo build --examples.

Then, run the Python RPC math server from the examples directory:

$ python3 server_rpc_math.py 
Connected: '{'session present': 0}', 'Success', '[ReceiveMaximum : 20, TopicAliasMaximum : 10]
Subscribing to math requests

Then run the Rust client to call it. Ask it to add a few numbers together:

$ ./target/debug/examples/rpc_math_cli add 1 2 3 4 5
15

You should see the request received and processed in Python:

requests/math/add  b'[1.0,2.0,3.0,4.0,5.0]'
Sending response 15.0 on 'replies/math/7DAEDF57-4259-87AD-90EB-9CA507DDB9E5': b'1'

Then maybe run the Rust service and call from Python, etc. All the sources are there to be examined.