Discovery using MQTT?

[JKRhb]

Currently, there is no possibility to discover TDs of devices using MQTT. I think having this feature, however, would be great to be able to use WoT TD in more scenarios and to be able to maybe even extend existing solutions like Home Assistant or zigbee2mqtt.

As MQTT allows for the use of wildcards when subscribing to topics, I could imagine that an MQTT consumer would subsribe to a topic like +/wot/thing_description while MQTT things would then publish their TDs to topics following a pattern like <divice-name>/wot/thing_description. I guess to reduce the workload for an exposed thing the TD should be published using the retain flag (which tells the broker not to discard the message after sending it to subscribers, but to keep it) and cleared using a LWT (Last Will and Testament) message.

Do you think this would be a useful addition to the specification?

[farshidtz]

I think specifying the topic and retainment logic would be a useful addition.

Regarding the topic, we should leave the ID in the end of the topic, because TD IDs are URIs and may contain slashes. If we use a template like wot/td/<id>, the wildcard topic wot/td/# will match both wot/td/x and wot/td/x/y.

For retained messages, there will be the problem of populating a broker with retained TDs of dead things. We should combine retained with a message expiry interval, added in MQTT v5.

[JKRhb]

Regarding the topic, we should leave the ID in the end of the topic, because TD IDs are URIs and may contain slashes. If we use a template like wot/td/<id>, the wildcard topic wot/td/# will match both wot/td/x and wot/td/x/y.

Looks like a very elegant solution to me!

For retained messages, there will be the problem of populating a broker with retained TDs of dead things. We should combine retained with a message expiry interval, added in MQTT v5.

Great, I wasn't actually sure if an expiry interval exists. This is much better than using an LWT message for removing the TD from the broker. However, maybe an LWT could be suggested as a solution for devices using a version < 5.0?

[farshidtz]

Great, I wasn't actually sure if an expiry interval exists. This is much better than using an LWT message for removing the TD from the broker. However, maybe an LWT could be suggested as a solution for devices using a version < 5.0?

True, setting a retained LWT (last will and testament) message to remove the retained TD is a good option for when clients disconnect ungracefully due to failure or decommissioning. But the clients need to remove the TD anyway before shutting down gracefully. Another issue is that we have to reserve the will of the client for removing the retained TD and this prevent its use for other purposes (e.g. removing retained measurements).

Keep is mind that not all clients remain connected all the time. It is common to have devices which wake up once a while to report measurements (and in this case a TD). For such devices, we can't rely on the connection-oriented will mechanism, but could use retained messages with expiry interval.

Writing recommendations for all these is a good idea.

[mmccool]

So only problem is that anything we add to a normative specification needs an implementation (in fact, two at least) before it can go to REC. We are close to having implementations of everything in HTTP but need two people to step up to implement any additional feature. I am a little concerned that we won't be able to fully spec MQTT (and CoAP) in time for the CR deadline also (and ideally we do the implementations before CR, so we can uncover any issues; making anything other than editorial changes after CR is basically not possible).

Perhaps we could create a separate informative document for this, experiment with it, and put it on the table for an update to Discovery next year, assuming we extend the charter for another round?

[JKRhb]

Hmm, how "mature" or extensive does an implementation have to be to meet the required criteria? I am currently working on two experimental projects (one of which is supposed to be an extension of an existing project) that would involve MQTT discovery. Is the main purpose to have a proof of concept/prototype or should the results be good enough to already be used in production?

If the latter is the case there might be some other problems with the MQTT (protocol bindings) specification and missing features in node-wot that could hinder a complete MQTT implementation (for example, with regard to read and write operations on properties).

If only the discovery itself has to be implemented I would assume that this could be achievable if the time frame is not too narrow. How would we proceed in this case? Maybe we could compose a list of required features (also with regard to different versions of MQTT) and then turn them into a PR?

Otherwise, should time be too short, the informative document also sounds like a good idea and might be the "safest" solution.

PS: Regarding CoAP (but also asked in general): Is there a list of features and their coverage by existing implementations?

[mmccool]

I think two "experimental" implementations would be enough. However, they have to come from different implementors. They can share libraries and such but should also be "different" code bases.

[JKRhb]

I think two "experimental" implementations would be enough. However, they have to come from different implementors. They can share libraries and such but should also be "different" code bases.

Thank you for your feedback on this! I almost finished a first prototype of an extension for zigbee2mqtt now which publishes TDs of paired devices to wot/td/{{DEVICE_NAME}}.

So far, it only really works with one device (a Zigbee lightbulb by Osram) as I haven't created more Thing Models yet (and I don't have any other Zigbee devices to test the implementation with at moment). In any case, it is already possible to use the emitted Thing Description with node-wot to toggle the lightbulb. I haven't implemented an interval for expiring retained messages yet but I will add this as an configurable option that can be enabled if the bridge is using MQTTv5.

By when do the implementations have to be finalized? And do they have pass some kind of formal procedure?

PS : I think the implementation can also be very interesting as a field for experimentation with regard to Thing Models as many of the supported devices either share common, extendable base or make use of certain capabilities which could be referenced or imported.