v5 properties possibly leaking into the payload on incoming messages

fpagliughi commented 2 years ago

Hey! I'm sending RPC requests with a bunch of v5 properties. A request looks like this:

$ mosquitto_sub -V 5 -v -t '#' -F "%j %X" --pretty
{
    "tst":  "2022-08-05T15:51:55.189543-0400",
    "topic":    "requests/sample/v1/test/zero_args/0",
    "qos":  0,
    "retain":   0,
    "payloadlen":   22,
    "properties":   {
        "content-type": "application/json",
        "response-topic":   "replies/MqttStubClient/259288/0ed0eaa8-14f8-11ed-9ae3-256dff621944",
        "correlation-data": "0",
        "user-properties":  {
            "timestamp":    "1659729115.1889336",
            "content-encoding": "gzip"
        }
    },
    "payload":  "\u001f�\b"
} 1F8B0800DB74ED6202FF8B8E050029BB4C0D02000000

The payload is the gzip'ed form of an empty JSON array: '[]', which encodes as "\x1F\x8b..." Both Wireshark and mosquitto_sub confirm the proper payload. (Ignore for the moment that the "compressed" payload is 10x the size of the original!)

On receipt with the Python client, it's reporting a payload of length 23 (instead of 22) as:

70.1f.8b.08.00.23.73.ed.62.02.ff.8b.8e.05.00.29.bb.4c.0d.02.00.00.00

There appears to be an additional byte at the beginning of the payload. It's a 0x70, which is "p" - the last letter in the last user property just before the payload starts in the PUBLISH packet (i.e. the "p" in "gzip").

It does, however, appear to properly decode the properties, reporting them as:

[ContentType : application/json, ResponseTopic : replies/MqttStubClient/259288/0ed0eaa8-14f8-11ed-9ae3-256dff621944, CorrelationData : b'0', UserProperty : [('timestamp', '1659729115.1889336'), ('content-encoding', 'gzip')]]

I'm working on an x86_64, Linux Mint 20 (Ubuntu 20.04), with released Python client 1.6.1 using Python 3.8

fpagliughi commented 2 years ago

This will create and send a message that get mis-read:

#!/usr/bin/env python3

import gzip
import time
import paho.mqtt.publish as publish
import paho.mqtt.client as mqtt
from paho.mqtt.packettypes import PacketTypes

payload = gzip.compress(b"[]")

cli = mqtt.Client(protocol=mqtt.MQTTv5)
cli.connect("localhost", 1883, 60)

props = mqtt.Properties(PacketTypes.PUBLISH)
props.ContentType = "application/json"
props.ResponseTopic = "replies/MqttStubClient/259288/0ed0eaa8-14f8-11ed-9ae3-256dff621944"
props.CorrelationData = b"0"
props.UserProperty = ("timestamp", str(time.time()))
props.UserProperty = ("content-encoding", "gzip")

cli.publish("test", payload, 1, False, props)

cli.loop(1.0)
cli.disconnect()

fpagliughi commented 2 years ago

Oops... looks like I had an older version of the package hidden in my path. Seems this was already fixed in the latest release. Never mind.

eclipse / paho.mqtt.python

v5 properties possibly leaking into the payload on incoming messages #672