[feat] Support new "in-progress" native Z-Wave MQTT Integration for HA

[Fishwaldo]

Not sure if you are aware, but Home Assistant is working with me on a new native Z-Wave Integration that replaces the old python-openzwave wrapper.

This new integration, like yours, is based upon MQTT as a transport, but is written to directly support Z-Wave without going through the MQTT generic device discovery layers (and subsequent challenges that you face).

I have no idea how hard it would be for you to implement a "alternative" JSON over MQTT schema, but if its not, it might eventually save you some time and headaches dealing with the MQTT discovery etc right now

This is very early alpha, but some initial success reported. You can view the actual integration here: https://github.com/cgarwood/homeassistant-zwave_mqtt and ozwdaemon (living at https://github.com/OpenZWave/qt-openzwave) are the implementations we have done so far.

The Schema of the JSON messages is roughly documented in the docs/mqtt.md file at the qt-openzwave repository).

[robertsLando]

I dind't get it @Fishwaldo

Not sure if you are aware, but Home Assistant is working with me on a new native Z-Wave Integration that replaces the old python-openzwave wrapper.

Nope didn't know

This new integration, like yours, is based upon MQTT as a transport, but is written to directly support Z-Wave without going through the MQTT generic device discovery layers

If your integration natively supports zwave it will be an alternative to mine? The only 'pro' to use mine is to have it separeted from hass, I should send messages to that in the format described in MQTT.md docs and it will automatically create/handle all hass entities ?

[Fishwaldo]

It’s a alternative to yours but we used a topic structure and payload that’s different to the current MQTT adapter in HA.

If you can duplicate the structure and payload (which is basically just exposing the existing values and attributes as a JSON payload, you don’t have to map to the different device types. The integration (in cgarwood’s repository) will handle all that.

Makes the Integration very simple from OZW side and the heavy lifting is done on HA side.

[chilicheech]

So, what value does this add? If z2m implements the same mqtt structure as HA's native zwave mqtt implementation then there's no need for z2m at all. The reason I switched to z2m is because sometimes home assistant makes assumptions about how things should be implemented without giving users the option to do it differently. z2m gives us more flexibility in that i don't have to follow HA's native implementation details. (Apologies if I misinterpreted this request 😄 )

[robertsLando]

If z2m implements the same mqtt structure as HA's native zwave mqtt implementation then there's no need for z2m at all

Yes that is what I mean

The only 'pro' will still be the fact that if you restart hass z2m will run in a separete process and you don't need to wait for a completely new network scan

[LordMike]

I'll note that I kinda prefer now to have z-wave running separately. When I do changes on HASS - it's a lot at once which often requires restarts .. having z2m nullifies the z-wave impact. :)

[mbhoek]

AFAIK the qt-openzwave component part of the new Z-Wave integration also runs separately from Home Assistant, meaning restarting HA would not impact Z-Wave as the current implementation does.

It is somewhat confusing to me why there are now two Zwave/MQTT bridges under the OpenZWave umbrella, which at the moment don't seem to be compatible. Any thoughts on what it means for the future of Zwave2Mqtt @robertsLando and/or qt-openzwave @Fishwaldo ?

[robertsLando]

@mbhoek Sincerly I don't know the main reasons behind qt-openzwave

[Fishwaldo]

This has been extensively discussed on the home assistant forum. Suggest you read there.

In essence - the existing "Generic" MQTT adapter in Home Assistant is just that - Generic and as I'm sure @robertsLando can attest to - Limited in functionality when "mapping" zwave devices to MQTT Generic Devices.

The New ozwdaemon and zwave_mqtt integration is only using mqtt as a transport. It does not try to "standardize" on any specific MQTT Schema and exposes full Z-Wave Functionality to Home Assistant.

The reason I invited @robertsLando to explore this new Home Assistant Integration is I'm a firm believer of competition. :) some people might appriciate the Web GUI of Z2M, others might prefer the barebones approach of ozwdaemon. Give the users a choice.

Final note - the OpenZWave organization in github is only there to group projects that utilize the openzwave core library in one way or another so its easier for people to find OZW projects. What @robertsLando does with z2m is completely upto him, just like other projects such as the dotNet, python or nodeJS wrapper authors can take their projects any direction they want. (or abandon them if they run out of time unfortuantly!)

[robertsLando]

@Fishwaldo I think that if we focused on a single project (z2m or another) we could have just one good project instead of multiple ones (that does same things) to maintain, this is my idea of opensource but I respect your decision

the existing "Generic" MQTT adapter in Home Assistant is just that - Generic and as I'm sure @robertsLando can attest to - Limited in functionality when "mapping" zwave devices to MQTT Generic Devices.

I agree on this, making mqtt discovery is a pain and there are many limitations. So I completely agree with this: https://github.com/cgarwood/homeassistant-zwave_mqtt but I don't understand the reason behind ozwdaemon as z2m was already existing and we could work together for the support of the homeassistant-zwave_mqtt plugin

[LordMike]

Given the ozwdaemons more hardcore/direct z-wave nature, I think it's a good addition. It seems to add some topics or details Z2M doesn't have (I didn't do a comparison)... I could be wrong. I do like Z2M's interface though. Will ozwdaemon do more advanced stuff, like be able to configure association groups?

But it seems that for both projects, we'll need to make a set of HASS discovery mappings for specific devices, like what ozw has in configs for specific devices.. :P

[robertsLando]

like be able to configure association groups?

You already can do this in Z2M

[LordMike]

I know there's UI for it. Question was for ozwdaemon and its MQTT-only nature. Because as long as ozwdaemon doesn't expose all the details of z-wave over the MQTT transport, it won't be able to stand alone.

[robertsLando]

@LordMike For that kind of things @Fishwaldo is working on a https://github.com/OpenZWave/ozw-admin that is the 'control panel' for OZW

[mbhoek]

@Fishwaldo Thank you for your explanation. I did read the HA forum before commenting on this issue, especially "The Future of Z-Wave in HA - QT-OpenZWave" and "Do I jump over to ZWave2MQTT or wait for QT-OpenZWave?". I also looked at https://github.com/cgarwood/homeassistant-zwave_mqtt to try and understand the differences between the two ZWave/MQTT bridges.

Seems to me the idea is to have ozwdaemon (qt-openzwave) 'dump' the raw zwave data onto the mqtt bus without any pre-processing, and have HA Core (homeassistant-zwave_mqtt) sort it all out and do the actual integration. Whereas Zwave2Mqtt does do some pre-processing of the zwave data, and for HA tries to come up with a mapping that can be used by homeassistant.mqtt.discovery to do the integration (please correct me if I'm wrong @robertsLando).

Why I commented on this issue and asked about the future of the projects was because in my mind there is no value in having 2 projects dump the exact same unprocessed zwave data onto the mqtt bus. So I figured there must be something I was missing. FWIW, I like that Zwave2Mqtt does some pre-processing because it makes it easier to integrate with other (non-HA) components (e.g. node-red).

I'll go back into lurker mode now, thanks for your time.

[Fishwaldo]

But it seems that for both projects, we'll need to make a set of HASS discovery mappings for specific devices, like what ozw has in configs for specific devices.. :P

This shouldn’t be necessary for the ozwdaemon as on the HA side it’s a pure Z-Wave implementation. On the discovery side that z2m it’s generic - made to support a wide range of protocols etc.

[Fishwaldo]

I know there's UI for it. Question was for ozwdaemon and its MQTT-only nature. Because as long as ozwdaemon doesn't expose all the details of z-wave over the MQTT transport, it won't be able to stand alone.

It exposes the entire OZW api. (At least those not marked for depreciation) - If it doesn’t have something - it will!

[Fishwaldo]

FWIW, I like that Zwave2Mqtt does some pre-processing because it makes it easier to integrate with other (non-HA) components (e.g. node-red).

I believe that was the original intent of Z2M. Home Asssistant discover support only came along later.

Ozwdaemon‘s MQTT support was written specifically for HA’s usecase. (It also does other things like allow concurrent access to the Zwave Network by the ozwadmin - or any app that uses the QT bindings in the same repository)

As to why i didn’t join forces with @robertsLando - I know nutz about nodeJS and he isn’t familiar with c++.

One thing I do think should be addressed is the naming of the new integrations. Zwave_mqtt is very easily confused with Z2M.

[robertsLando]

Thanks for the explanation @Fishwaldo :smiley:

[LordMike]

Just putting it out there ... one wild idea would be to adapt Z2M to work, not with the OZW api, but with the MQTT topics provided by ozwd.... :O

[robertsLando]

@LordMike As I said I'm completely +1 with https://github.com/cgarwood/homeassistant-zwave_mqtt

HAss mqtt discovery is a real pain and limited in functionality. I just need to create a secondary gateway that sends messsages in the format that plugin is expecting

[fredless]

Zwave_mqtt is very easily confused with Z2M.

and when you consider that there is also a zigbee2mqtt project out there... I seriously need a decoder ring.

[Fishwaldo]

In the next release of HA, the new integration has been renamed.

[rotem925]

I would have a look at this: https://github.com/home-assistant/architecture/issues/483 Something is cooking...

[robertsLando]

@rotem925 This: https://github.com/zwave-js/zwave-js-server will be embedded in https://github.com/zwave-js/zwavejs2mqtt

And a new addon will be created on hass side. This will replace qt-openzwave and ozw beta addon

[rotem925]

@robertsLando Thanks for the update on this.. I'm now using ozwdaemon. Is this ready yet? It is going to be a nightmare to migrate everything all over again..

[robertsLando]

Not ready yet

[rotem925]

@rotem925 This: https://github.com/zwave-js/zwave-js-server will be embedded in https://github.com/zwave-js/zwavejs2mqtt

And a new addon will be created on hass side. This will replace qt-openzwave and ozw beta addon

Thank you @robertsLando !! Do you know if the HA integration plan will talk directly to the zwave-js-server? If that the case, doesn't it mean that zwavejs2mqtt is not necessary? Sorry, I just a bit confused. I have a lot of issues with ozwdaemon, and this time I want to make a better choice :) Not having to rename every time I switch..

thank you!

[robertsLando]

zwavejs2mqtt is not necessary

Nope, zwavejs2mqtt will come with zwave-js-server inside, it will be used as control panel too and also you can use it to manually discover some entities when zwave-js-server fails to create some

[rotem925]

@robertsLando Got it! I will give it a try then. Thank you!