Feature Request: zwavejs2mqtt support

[adamcstephens]

Would it be possible to add zwavejs2mqtt support to this exporter?

https://zwave-js.github.io/zwavejs2mqtt/#/usage/setup?id=gateway

[kpetremann]

hi @adamcstephens,

Sure I can work on that. But I don't have any zWave device. Could you help by providing real life examples so I can work from tests? I can work from the doc you provided, but I would prefer real life examples.

[adamcstephens]

Yep! I'm happy to help however best I can. Here's a dump of topics/values in my environment with the two automatic topic types.

named topics - https://gist.github.com/adamcstephens/4bf4a312cbaec0e3c5abb6a23271eb8d valueID topics - https://gist.github.com/adamcstephens/9c4e3a144d9734913809b560cb0851da

[adamcstephens]

This is a lot to unpack. An alternative would be to add Home Assistant mqtt discovery support. https://www.home-assistant.io/docs/mqtt/discovery/

zwavejs2mqtt can publish in this format too https://zwave-js.github.io/zwavejs2mqtt/#/usage/setup?id=home-assistant

One benefit of using the Home Assistant format is compatibility with other producers (see list) at the cost of some of losing access to some of raw zwave data. The added compatibility is worth the tradeoff for me and still meets my needs.

[kpetremann]

Thanks for the input data it will help.

I would prefer work on the data source, and not depend on Home-Assistant. While it is a great tool (I use it), there are alternatives and I don't want to prevent people from using this exporter if they don't use HA.

However, this is a good idea. I can work on adding both HA compatibility format, and zwave-js, in dedicated Feature Requests :)

I'll focus here on the zwave-js integration.

[adamcstephens]

I completely agree that this should not depend on HA, I only meant to suggest adding the format support. I actually don't want to use HA, which is one of the reasons I'm interested in this exporter. :)

[kpetremann]

I only meant to suggest adding the format support

yep, we are perfectly aligned ;)

[kpetremann]

can you test with this branch? (It may need improvements)

Don't hesitate to provide the Prometheus metrics it exposes.

[kpetremann]

Actually I have to rethink the integration, to better respect the format. I need to find a way to differentiate named from valueid topics (because I ma targeting supporting only named topics).

I am a little confused as your examples does not show property_key while in the doc it is not marked as optional

[kpetremann]

nevermind, I had a look to ZWavejs2mqtt source code to find answers :) I've update the branch, please test :)

[adamcstephens]

Gave it a try with my setup and it crashes with the below output. I'm currently on valueid, and will switch to named for this support, but I need to update some automations first. I'll get that done (hopefully this evening) and try again, but I figured a crash is probably undesired anyway.

app_1  | INFO:mqtt-exporter:subscribing to "#"
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_currentTrack_ItemId
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_currentTrack_ParentId
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_enqueuedMetadata_ItemId
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_enqueuedMetadata_ParentId
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_ts
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_volume_Master
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_volume_LF
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_volume_RF
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_mute_Master
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_mute_LF
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_mute_RF
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_bass
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_treble
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_connected
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_state
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_currentvalue
app_1  | INFO:mqtt-exporter:creating prometheus metric: mqtt_targetvalue
app_1  | Traceback (most recent call last):
app_1  |   File "/opt/mqtt-exporter/exporter.py", line 5, in <module>
app_1  |     main()
app_1  |   File "/opt/mqtt-exporter/mqtt_exporter/main.py", line 299, in main
app_1  |     client.loop_forever()
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
app_1  |     rc = self._loop(timeout)
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1164, in _loop
app_1  |     rc = self.loop_read()
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1556, in loop_read
app_1  |     rc = self._packet_read()
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 2439, in _packet_read
app_1  |     rc = self._packet_handle()
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3033, in _packet_handle
app_1  |     return self._handle_publish()
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3327, in _handle_publish
app_1  |     self._handle_on_message(message)
app_1  |   File "/usr/local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3570, in _handle_on_message
app_1  |     on_message(self, self._userdata, message)
app_1  |   File "/opt/mqtt-exporter/mqtt_exporter/main.py", line 249, in expose_metrics
app_1  |     topic, payload = _parse_message(msg.topic, msg.payload)
app_1  |   File "/opt/mqtt-exporter/mqtt_exporter/main.py", line 213, in _parse_message
app_1  |     topic, payload = _normalize_zwave2mqtt_format(raw_topic, payload)
app_1  |   File "/opt/mqtt-exporter/mqtt_exporter/main.py", line 193, in _normalize_zwave2mqtt_format
app_1  |     payload_dict = {properties.lower(): payload["value"]}
app_1  | KeyError: 'value'

[kpetremann]

totally, it is not supposed to crash but just discard it. Thanks for the output.

[kpetremann]

fix pushed, any better?

[adamcstephens]

Yes, that's better. It no longer crashes.

I'll get you a dump of the metrics soon. I want to drop all my retained messages and get a clean capture of the named topics, and I've unfortunately not gotten to that yet.

[adamcstephens]

zwave data: https://gist.github.com/adamcstephens/09770a8b731226a752700b1b779458df

exporter data: https://gist.github.com/adamcstephens/34d885bc67ad7be1c695465029f17080

[kpetremann]

there are tons of metrics, maybe a lot of them are useless. However, it seems the relevant metrics are working, for instance: mqtt_air_temperature{topic="zwave_backroom_multisensor_sensor_multilevel_endpoint_0"} 83.2

What do you think? is there anything missing?

[adamcstephens]

Yeah, most of these metrics (and the corresponding mqtt messages) aren't useful for me. Some filtering would be nice, but isn't critical.

I'm not seeing anything missing at this time, so this looks good!

[kpetremann]

great!! :tada::tada::tada:

To filter out the metrics you don't want, don't hesitate to leverage MQTT_IGNORED_TOPICS option.

I'll merge the branch right away and close this feature request.

Don't hesitate to open issues if you have others feature requests or issues, I'll be happy to help.

Thanks

[adamcstephens]

Thank you!