Timestamp on MQTT messages during `on_publish`

[kushwiz]

To help us save time and help you faster

Looking to add dynamic timestamp on MQTT messages during on_publish. The reason is we use Shared Subscriptions and there is no way to maintain order of state messages. Even if we add timestamp from source, the LWT message that actually holds the ERROR message has no timestamp.

Please do not insert images of text, but insert the text instead.

Environment

• VerneMQ Version: 1.10.0
• OS: Debian 9.11 (erlio/docker-vernemq)
• Erlang/OTP version (if building from source):
• VerneMQ configuration (vernemq.conf) or the changes from the default
• Cluster size/standalone: 3 containers

Expected behavior

Looking to add dynamic timestamp on MQTT messages during on_publish. The reason is we use Shared Subscriptions and there is no way to maintain order of state messages. Even if we add timestamp from source, the LWT message that actually holds the ERROR message has no timestamp.

Actual behaviour

I don't find a way to do that.

[kushwiz]

I tried using the on_deliver hook, It is very close to what i am looking for. But I am trying to understand where exactly it sits in the entire lifecycle of a MQTT Message and its Topic. I need some help with that.

[ioolkos]

Yes, you need a hook that can modify the payload (if your point is about adding the timestamp to the payload). on_deliver has that: https://github.com/vernemq/vernemq_dev/blob/master/src/on_deliver_hook.erl

There's documentation on the Publish Flow here: https://docs.vernemq.com/plugindevelopment/publishflow and I just noticed that the flow diagrams are not rendered. This one should be on the page: https://github.com/vernemq/vmq-docs/blob/master/.gitbook/assets/publish_flow.png

I'm sure you're aware that timestamps only cannot guarantee order of message delivery in a distributed system.

Note that the newest Docker builds are now always here: https://hub.docker.com/r/vernemq/vernemq

[kushwiz]

Yes, timestamps only will not guarantee. But since these are only state messages, we only care about the latest state of our device.

Does the on_deliver hook run FIFO on a per topic basis?

[ioolkos]

In general MQTT considers topics as "ordered topics", yes. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718105 There's a lot of implications, QoS, inflight windows, message Dup flag.

Message order is not guaranteed when you have shared subscription over multiple cluster nodes. So it seems, if at all, a timestamp at the device source is the one that would help? on_deliver is just a hook called on every node. It cannot be coordinated cluster-wide to ensure in-order delivery.

[kushwiz]

Ok. on_deliver hook will not work for my use case.

I can send timestamp from source, but how will LWT be handled in this case? I cannot dynamically set %timestamp% in my LWT message.

[kushwiz]

It would be great if I could mutate the payload using the on_publish hook. Hopefully that would let me mutate LWT messages

[ioolkos]

on_publish is only a notification hook. You could change the payload in auth_on_publish.

[kushwiz]

I understand. My concern is that LWT does not trigger the auth_on_publish hook

[ioolkos]

Correct, a Last Will publish will not trigger auth_on_publish (it got already authorised at connect time). It will trigger on_publish but then you cannot change the payload.

[kushwiz]

Yes, FYI - LWT does not trigger on_publish.

[ioolkos]

Note that the LW message will only be sent for disgraceful disconnects. When the clients sends a DISCONNECT packet, the LW message will not be sent.