[FEATURE REQUEST] Forward all value updates to MQTT

[escoand]

Is your feature request related to a problem? Please describe. The current MQTT solution is just transmitting some manual selected parameters regularly to the broker.

Describe the solution you'd like I would like to propose to implement a way to constantly publish all BSB/LSP values to the MQTT broker. Furthermore I would like to add MQTT discovery messages (https://www.home-assistant.io/docs/mqtt/discovery/). With this solution you don't need to add any configuration (beside the MQTT broker) in your smart home system, at least Home Assistant. The overall goal for me is to use BSB-LAN just as a bridge from BSB/LSP to MQTT and do all the view, historize and controlling stuff with Home Assistant.

Describe alternatives you've considered I've implemented a working solution based on ESPHome. But this is currently only listening on BSB, so manly only the "Kesseltemperatur" is updated regularly in my case. https://gist.github.com/escoand/180b6b317705944887c6fcdae3007a16

Describe your own contribution I could adapt my implementation to BSB-LAN. The issue is mainly for discussion the general idea and the concrete implementation.

Additional context This was already initially discussed with @fredlcore and @1coderookie via mail.

[jbaudoux]

I can make some tests (more likely over the week-end) and provide you some feedback. It would have been better if you had opened a PR on which we could have reviewed & commented.

[fredlcore]

@mirkolenz: Oh, then I don't want to give you any kind of excuses for procrastination ;) - which I did probably six out of my seven years of writing on my PhD ;). I guess it's just a minor thing, and worse comes to worst, I'll just drop the temperature thing and just send it as all the other numerical parameters, too.

@jbaudoux: What do you mean I should have opened a PR? So I can wait until I get your blessing for merging my own PR in my own project? You can comment on any commit in the project's history if you like, but given the fact that I'm still waiting for answers and explanations I asked for in the PRs that you have submitted, I guess you should focus on these first before lecturing me on what is proper procedure.

[fredlcore]

Ok, bug is fixed, there was no icon for mdi:temperature, but for mdi:thermometer. Few other bugs also fixed and tested various parameters. Works fine here, so we can finally close this request after two years :). Thanks to the helpful contributions, especially @mirkolenz, and @liudger for the initial BSB-LAN implementation!

[fredlcore]

Forgotten to mention: URL command /M1 creates devices, /M0 sends remove messages.

[mirkolenz]

Great news 👍👍 (also nice to hear that others are also faced with procrastination during their PhD 😊)

And just to clarify: Did you implement the mentioned workaround of publishing parameter values to MQTT when receiving an HTTP request? That could be useful for me in the future.

I will let you know if the new feature is capable of replacing my manual setup as soon as I have the time to tinker with it.

[jbaudoux]

And just to clarify: Did you implement the mentioned workaround of publishing parameter values to MQTT when receiving an HTTP request? That could be useful for me in the future.

It's here https://github.com/fredlcore/BSB-LAN/pull/641

[fredlcore]

@mirkolenz: Yes, I have. Every time a query for a parameter is being made (and that includes SET commands since the result is queried here as well after setting it), the result gets forwarded to the MQTT broker. The only issue I'm having is with the error status parameters (6800++). HA keeps saying "Unknown", although these are defined as ordinary (read-only) sensors, so they should just display the payload, same as all the other parameters. Now they have even disappeared from the dashboard, but are still listed under "Settings"/"Devices"/"MQTT"/"Entities". Strange...

But other than that it's really cool to see how just by configuring the MQTT broker in BSB-LAN and HA, I now can just call /M1and hundreds of parameters (mostly with appropriate icons based on their unit of measurement) pop up, and then I just do /0-9999, and quickly everything fills up with the proper values - nice :)...

Just one last question regarding retaining messages: Should I turn this on by default in the auto discovery messages or just when sending the values (or both)?

[mirkolenz]

Thanks for the clarification!

Regarding the message retainment: I think the best practice is to retain only the availability of the bridge and not individual values. Especially when taking into account rhetoric potentially large number of parameters that may be published to the broker, I do not know how the retainment could affect performance. So I would just set this to "off" for all parameter updates. Does that make sense to you?

[fredlcore]

Yes it does, thanks, and that is how I configured it at the moment. With "bridge" you mean the BSB-LAN device, right? That already sends the "online" and "offline" status as a retained parameter IIRC. So we can then just leave things as they are, right?

[fredlcore]

By the way: What should also work now is that whenever there is a change made via the room unit or the display on the heater, the new value would also be sent automatically to MQTT (however the mentioned problem that BSB-LAN may miss this change due to performing other tasks at the same time still applies). I have no means to test this, so if you could at some point check this out, that would be great.

[mirkolenz]

With "bridge" you mean the BSB-LAN device, right?

Correct

That already sends the "online" and "offline" status as a retained parameter IIRC.

Yes that was what I mean, I did not remember the correct terms 😄

So we can then just leave things as they are, right?

Also true!

By the way: What should also work now is that whenever there is a change made via the room unit or the display on the heater, the new value would also be sent automatically to MQTT (however the mentioned problem that BSB-LAN may miss this change due to performing other tasks at the same time still applies). I have no means to test this, so if you could at some point check this out, that would be great.

And that would work even if the parameter is not listed in the logging parameters? That would be an extremely useful feature. I will report back if this does indeed work as intended once I flashed my ESP32 with the latest version of BSB-LAN.

And thanks for your hard work!

[fredlcore]

Your're welcome :). Yes, any query (i.e. ´/8700/8830´ or ´/0-9999´) as well as any SET command sent to the heater (either from BSB-LAN or from a room or heating display unit) will also be sent to MQTT. But as mentioned, the latter cannot be guaranteed to be "caught". Logging parameters might still be useful for parameters that you really want to have logged in a specific interval without setting up something on your home automation system, but with the latter, you are much more free to implement different logging intervals (which was also one thing that many users asked for). In case catching a SET parameter does not work as intended, please attach a SerMo log and if possible also a log what comes in on the MQTT broker. But I hope it'll just work out of the box :)...

[jbaudoux]

Regarding the message retainment: I think the best practice is to retain only the availability of the bridge and not individual values.

Any MQTT state value should be retained by the broker. Commands should not be retained. When you start home assistant, it subscribes and initializes display with retained value. The broker acts as a state cache. BSB informs the broker of any state change. If you don't do that, I don't see how it can work properly

[mirkolenz]

Yeah that makes more sense than my statement. Thanks for pointing out the difference between state and control values!

[fredlcore]

So I still set all values I send to the broker to "retained" because also the values from the SET commands should be retained as they are the new values for that parameter, right?

[fredlcore]

Sorry, I just realized that I hadn't pushed the changes that also catch the SET commands to the master repo. So in case you have already downloaded a previous version, please do so again with the most current master.

[mirkolenz]

So I still set all values I send to the broker to "retained" because also the values from the SET commands should be retained as they are the new values for that parameter, right?

Yes I think so. You should basically set the retain flag for every parameter update. So if receiving a SET command, you would respond with the new value and retain it so new clients can immediately use the provided value instead of having to wait for the next log interval and/or state change.

[fredlcore]

Alright, I've pushed this feature to GitHub now. Once you've tested whether setting a parameter on a room unit really works, we can consider this one really done :)...

[mirkolenz]

Great, thank you!

[fredlcore]

No pressures, but does this mean you tested the SET feature? If not, no hurries, just so I get it right :)...

[fredlcore]

One more question: There is a "dangerous" switch next to the general device name (in my case "BSB-LAN"). If I turn it off and on again, it writes all the current parameters to the heating system (by sending SET commands). This can really mess up your heating system if you use auto discovery, but do not fill (all) the parameters with up-to-date values. Is there a way to prevent hat the device that combines all the auto discovery parameters (i.e. "BSB-LAN") has such a switch? I can hardly think of scenarios where this would be useful or even usable, and I just spent 20 minutes restoring all the parameters from a "backup" (written down on paper ;) )...

[mirkolenz]

No pressures, but does this mean you tested the SET feature? If not, no hurries, just so I get it right

Sadly not, my plan still is to test it in the summer once all papers have been submitted 😃

Is there a way to prevent hat the device that combines all the auto discovery parameters (i.e. "BSB-LAN") has such a switch?

Not that I know... These global switches are also added to the automatically generated dashboard, this is one of the reasons I now have a custom one: To prevent accidentally setting all devices at once. If you find a way to remove the switch please let me know, would be interesting for me as well.

[fredlcore]

Ok, no problem, I'll ask in the forum. As for the global switch, we better add a big warning in the manual then...

[fredlcore]

@mirkolenz: I found this elsewhere, but I can't test it: https://community.home-assistant.io/t/how-do-i-get-rid-of-this-switch-toggle/103426/4 Could you confirm that disabling the group toggle switch can be removed this way and maybe send me a screenshot?

[mirkolenz]

@fredlcore Sorry for the delay, I am currently on a business trip. As far as I know, the setting show_header_toggle: false is only applicable to user dashboards and cannot be set by the MQTT integration. At the same time, I think the setting control: hidden is only applicable to custom entity groups. They also changed quite a few things related to group organization since 2019 (in the linked page there still exists documentation for "old-style" groups).

[fredlcore]

@mirkolenz No worries, whenever you're ready, you're ready :) - so that's a bit of a bummer that both settings would not help. But even if it is not usable via MQTT, would show_header_toggle: false work once the BSB-LAN entity has been added to the dashboard? Because that's where this global switch shows up, right? In the list of devices and entities, it doesn't show up, so there it's not a problem. Or am I missing something here?

[mirkolenz]

would show_header_toggle: false work once the BSB-LAN entity has been added to the dashboard? Because that's where this global switch shows up, right? In the list of devices and entities, it doesn't show up, so there it's not a problem. Or am I missing something here?

Yes that is totally possible, but that would require users to have custom dashboards. I am not aware of a mechanism to disable this behavior for the auto-generated dashboard where the entities are organized by devices.

[fredlcore]

Ah, understood, didn't know that there is a difference, thanks!