Add support for Homie MQTT convention

I was thinking of writing a Google Home-centric smarthome platform in Rust myself, and this looks quite similar to what I had in mind (from what I can see so far, some documentation would be good!), but it looks like you're using a custom protocol over websockets to communicate between devices and the server. It would be useful to support a standard protocol that works with existing devices, and the one that I prefer is the Homie convention, which defines a way to structure MQTT topics so devices can define their capabilities and be discovered automatically.

I already wrote a couple of Rust crates to handle this (homie-controller for the controller side that could be integrated into houseflow), which would do a fair bit of the work. We'd need to define how it maps to the Google Home API structure, or perhaps make it configurable.

I might be able to send a PR to add this, if you're interested and can help me understand the architecture a bit first.

Hi, Houseflow used MQTT in previous versions, but I found it very inefficient and impractical for few reasons:

Security and authorization

Here's an example MQTT topic: devices/afdc3985-8a79-4e7f-9dfb-2f9987b0e50a/light/set, the broker must ensure that no one except the server and device(afdc3985-8a79-4e7f-9dfb-2f9987b0e50a) can subscribe to this topic. There are two approaches that I know for authorization in brokers

Custom plugin. Mosquitto allows creating custom authorization plugins, see.
HTTP requests to pre-defined endpoint with question if the client can subscribe or not. That's how it used to be in Houseflow some time ago, see EMQX HTTP Auth plugin configuration.

If we use former, we're required to use C, though the language by itself is not a huge problem, much more problematic is the build system, I managed to write a dockerfile which made this a bit easier, though I am still not sure if it's correct, and that probably wouldn't work outside of docker to which I don't want to be tied up. For comparsion building Rust project is simply cargo build. Another problem with this approach is that I'd not be able to easily share libraries from existing codebase, so I'd have to reimplement reading all configurations or write a FFI wrapper which would complicate stuff even more.

If we use latter, we need to use EMQX, which takes super long to start up and shut down, from 20s to even 1 minute, that's too long for me and makes the development process much slower.

Simplicity

Broker is another dependency so that most likely requires Docker, and the broker is also ran in another process than server, so we also need to have a docker-compose to manage them, I don't want to be forced to do that. I want to keep stuff simple and make the server to be that low on resources so it can be ran on cheap Raspberry Pi, and docker is quite heavy, I'm using systemd service in production, but also planning to make a Dockerfile available. Currently houseflow server is a single process, with no other dependencies.

I recently migrated from PostgreSQL to SQLite to remove other process dependency(I can easily plug in another databases if needed, but SQLite seemed most suitable for this use-case). The database used only for storing users, devices are defined in configuration files, and it makes it easy to reproduce in other environments.

No sense in using MQTT for point-to-point communication

There's no much sense to use MQTT in point to point communication, websockets fits much better in this use-case, and it gives more control.

It'd be great o hear other opinions about this project and my architecture, I'll finish the documentation by this weekend. If it's easier we can talk in matrix @gbaranski:matrix.org.

Thanks for the reply!

I'm not sure I understand your security model. Why must the broker ensure that no one except the server and device can subscribe to the topics for a device? As far as I'm concerned, the household is the security boundary I care about. If someone who manages to compromise one of my lightbulbs can use its credentials to turn another light on and off, that doesn't seem like much of an issue, as they are presumably already in my house by that point. So I'm happy for all the devices in my house to connect to the MQTT broker with the same credentials, as long as they are kept separate from devices in someone else's house.

This makes authentication pretty simple; I just have Mosquitto configured to allow full access to all topics under a certain prefix to the matching user. (My setup is a bit more complex in that I have another user with read-only access for saving historical data, but there's still nothing terribly complicated.)

I'm not familiar with Docker, but I'm not sure why that would be required. Mosquitto is available in Debian (and I guess other MQTT brokers too), so it's pretty easy to install and configure as you like, on a Raspberry Pi or otherwise.

I agree that keeping things as simple as possible is good, and switching to SQLite sounds like a reasonable idea, but I think that using a standard protocol simplifies things by letting you reuse a lot of existing code rather than having to reimplement everything from scratch. If you invent your own protocol, then you have to write software supporting that protocol for each device you want to connect.

It might be helpful to give an overview of my current setup, to give an idea where I'm coming from. I currently have:

A VPS, running:
- Mosquitto, listening on Websockets and raw MQTT over TLS, configured with multiple user accounts where each user can access a sub-tree of topics.
- InfluxDB, with a database per user
- homie-influx, which sends a data point to InfluxDB whenever the state of some node changes (to the matching database for the user).
- OpenHAB, configured (somewhat manually) with all the devices, exposing them to Google Home via https://www.myopenhab.org/.
In my house:
- Some light controllers, connecting to the MQTT broker.
- A couple of Raspberry Pis running mijia-homie to connect to a bunch of BLE temperature/humidity sensors and report their readings over MQTT.
- Sometimes, some other software to bridge a Bluetooth smart barbecue thermometer to MQTT.

I can then use https://rroemhild.github.io/hodd/, for example, to give me a web UI to see and control everything, and I can easily add new devices as long as someone has written some Homie-compatible firmware or bridge for them (of which there are fair few already).

I'm fairly happy with most of this, but the annoying part is OpenHAB, which requires a lot of manual configuration, is quite heavyweight and annoying to install, and doesn't have any feasible way of supporting multiple users (i.e. providing a hosted services to my friends to use for their homes). The only thing I'm using it for at this point is to provide the Google Home integration. So what I'd like to have is a nice lightweight service that can handle multiple users, automatically discover all devices via Homie, and expose them to Google Home. I'm not sure how well this fits with your goals for Houseflow because I'm not really sure what your goals are, but perhaps it might also make sense for you. If not, perhaps we could still share code for the Google Home parts, or support running in different configurations or something. In any case, I'm happy to chat more. I'm not familiar with Matrix but I'll have a look, or if you are on Discord that might be easier.

This makes authentication pretty simple; I just have Mosquitto configured to allow full access to all topics under a certain prefix to the matching user. (My setup is a bit more complex in that I have another user with read-only access for saving historical data, but there's still nothing terribly complicated.)

That can work too, but it is not as integrated as my solution is, suppose that you want to support Google Home, you'd need to copy all permission related stuff from your mosquito configuration to some other place from which server can read just to make accepting SYNC intent possible.

I'm not familiar with Docker, but I'm not sure why that would be required. Mosquitto is available in Debian (and I guess other MQTT brokers too), so it's pretty easy to install and configure as you like, on a Raspberry Pi or otherwise.

Configuration yes, but making custom plugins require compiling plugin as a shared library and then plugging it, it's fine if you do it only once, but if you want to reproduce it on other computers easily, its much more complicated.

My current setup includes a 15 year old laptop running Houseflow server, two ESP8266 modules working as a Gate and Garage in my family's home.

It's very simple now, but I'm still waiting for DHT22 modules from AliExpress to plug them to ESP8266 or a Raspberry Pi

I'm fairly happy with most of this, but the annoying part is OpenHAB, which requires a lot of manual configuration, is quite heavyweight and annoying to install, and doesn't have any feasible way of supporting multiple users (i.e. providing a hosted services to my friends to use for their homes).

Houseflow is designed to support many users and houses, I had in mind that maybe someday I'll be able to sell Houseflow device to someone.

I'm not familiar with Matrix but I'll have a look, or if you are on Discord that might be easier.

Tell me your discord nickname and I'll reach you out.

Sure, I'm qwandor#0264 on Discord.

Coming to the conclusion, I think Homie MQTT Convention won't fit with current architecture. Maybe it'll make more sense after #238 will be merged. I'm closing this issue for now.

gbaranski / houseflow