Support streams or binary data

[rumbu13]

Currently Homie Convention allows only UTF8 encoded strings, but in order to keep the pace with the current devices, I thing there is a need for a binary data type.

We have cameras, speakers, players. Devices processing images, sound and voice are everywhere, therefore the most efficient way is to use a binary stream. Of course any binary stream can be base64-encoded and fallback to the string representation but this is far from efficient.

[Tieske]

I have no experience with it whatsoever, but how would a binary stream be passed around in MQTT?

[rumbu13]

In its core, mqtt works with binary data only, the fact that you choose to encode/decode that data in a string using utf8 is just a choice, mqqt protocol is not aware of that. Therefore the default data format you send/receive to/from a mqtt server are just simple bytes that you obtain through encoding/decoding a string.

That's why we can specify in the convention a new data type called "binary" or "bytes" where you can publish/subscribe raw bytes. It means basically that you don't need to utf8 decode the data found on a subscribed topic and take it as it is. It can be a picture, a sound, a video frame, a zip stream or even raw data captured from a device. The only limit imposed by mqtt protocol is the size of the packet - 256 MB.

What we can do in the convention is to make mandatory to publish also the mime type like image/jpeg, audio/vorbis, font/ttf. More than that, mqtt 5.0 allows to specify the mime type/format in the packet header.

[Tieske]

The binary data I get. But you mentioned supporting streams. How would that work?

[rumbu13]

Streams are a particular case of binary data, publishing a video frame at variable time intervals makes it a stream. What can be added for this particular case is for example a mandatory empty payload to signal the end of stream for subscribers.

For my particular use case (camera and voice), a binary data type in the convention should be enough.

[Tieske]

A binary type does make sense to me as well. You could propose a PR against the homie5 branch.

[Thalhammer]

While I agree with a binary type being a good idea, I don't think the use cases presented here are a good example. MQTT is binary safe, but its not a streaming protocol. Messages can get dropped (at QOS 0) or arrive multiple times in different orders (QOS 1). The maximum message size for a MQTT payload is ~250MB, but the broker is free to impose a lower limit. Most brokers do (AWS: 128Kb, Google Pub/Sub: 256Kb, IBM: 4Mb) and even if you run a broker that allows large payloads you start to hit another problem. MQTT doesn't support splitting the payload, meaning for the duration of the media upload there is no way for you to react to anything else. To make matters worse, you can't even cancel the upload without reconnecting.

TLDR: MQTT is not the right protocol for media streaming. It might work for small deployments and in your special case, but its definitely not going to work at scale. Use a protocol thats designed for that purpose instead (e.g. WebRTC) and exchange the data related to it via mqtt (e.g. peer information / endpoint infos).

[jacoscaz]

TLDR: MQTT is not the right protocol for media streaming. It might work for small deployments and in your special case, but it's definitely not going to work at scale. Use a protocol thats designed for that purpose instead (e.g. WebRTC) and exchange the data related to it via mqtt (e.g. peer information / endpoint infos).

This resonates strongly with my experience. It would be better to extend homie to model external data sources.