[FEATURE REQUEST] Forward all value updates to MQTT

[escoand]

Is your feature request related to a problem? Please describe. The current MQTT solution is just transmitting some manual selected parameters regularly to the broker.

Describe the solution you'd like I would like to propose to implement a way to constantly publish all BSB/LSP values to the MQTT broker. Furthermore I would like to add MQTT discovery messages (https://www.home-assistant.io/docs/mqtt/discovery/). With this solution you don't need to add any configuration (beside the MQTT broker) in your smart home system, at least Home Assistant. The overall goal for me is to use BSB-LAN just as a bridge from BSB/LSP to MQTT and do all the view, historize and controlling stuff with Home Assistant.

Describe alternatives you've considered I've implemented a working solution based on ESPHome. But this is currently only listening on BSB, so manly only the "Kesseltemperatur" is updated regularly in my case. https://gist.github.com/escoand/180b6b317705944887c6fcdae3007a16

Describe your own contribution I could adapt my implementation to BSB-LAN. The issue is mainly for discussion the general idea and the concrete implementation.

Additional context This was already initially discussed with @fredlcore and @1coderookie via mail.

[1coderookie]

Thanks @escoand , I really like this idea/function! :) @dukess and @Luposoft63 please note this issue..

Just a little note about your current state, that right now only the boiler temperature ("Kesseltemperatur") is transmitted: that's because it's the default parameter which is shown at the display of the controller unit at your heating system. The controller sends an INF message via broadcast with this parameter - if you have a look at the display of the controller unit, you'll see this parameter as a permament displayed parameter at the display. (If you'd have the adapter and the BSB-LAN software, you could see that telegram coming in from the controller regularly also at the serial monitor.) If you'd have an additional room unit like QAA75 for example, then also the room temperature ("Raumtemperatur") would be transmitted, because that is the standard parameter which is shown at the room unit.

If you want to test if your function is already working with different parameters, you can do it by simply quering another parameter at the display of the controller unit. Of course you'd need the adapter setup and the connection to the controller for that, but just to have it mentioned already.. E.g.: If you use the info button and switch to the outside temperature, the dhw temperature or so, the belonging parameter should also be sent over the bus and you should be able to receive it via MQTT then. This also works if you e.g. enter a menu and have a parameter displayed, which doesn't appear using the info button. So let's say you are entering the menu and you are picking the parameter for setting the dhw setpoint temperature, then that parameter will be send across the bus as long as it's been displayed. If no changes occur, the display switches back to the default parameter boiler temp / Kesseltemperatur after a while.

[escoand]

If you want to test if your function is already working with different parameters, you can do it by simply quering another parameter at the display of the controller unit. Of course you'd need the adapter setup and the connection to the controller for that, but just to have it mentioned already.. E.g.: If you use the info button and switch to the outside temperature, the dhw temperature or so, the belonging parameter should also be sent over the bus and you should be able to receive it via MQTT then. This also works if you e.g. enter a menu and have a parameter displayed, which doesn't appear using the info button. So let's say you are entering the menu and you are picking the parameter for setting the dhw setpoint temperature, then that parameter will be send across the bus as long as it's been displayed. If no changes occur, the display switches back to the default parameter boiler temp / Kesseltemperatur after a while.

Yes, this already working correctly. But my wiring is currently only connected to CL+ so no requests of the other parameters is possible.

[fredlcore]

I also like the idea, especially if it is zero-config later on :). What I don't understand is how you want to manage to send "all" parameters to the broker. On some heaters these are more than one thousand! Querying one parameter takes approx. 2-3 seconds (the bus only works on 4800bps plus the time it takes for heater to answer). So if you query let's say the 1600 parameters that the RVS63 offers, that would be more than one hour to query all of them. So you couldn't even get a stable one hour interval for each parameter - and with many of them, you'll want a much tighter schedule, such as every 5 minutes or even less.

But maybe I got this wrong and in the end, it's the Home Automation System which tells BSB-LAN which parameters to send, I don't know. We chose the 20(?) parameters you can define for pushing in BSB-LAN because that seemed to be a good compromise between polling interval and not blocking the bus too much. But I'm in favor of any solution that gives users the flexibility they want and at the same time the bus enough "air to breathe" :)...

[escoand]

@fredlcore As far as I can see is BSB-LAN providing a history for some/important/any values. They need to get polled from the bus as well, or how is this done? That's the same point I would additionally send the answers to MQTT. Nothing more, but also nothing less. So my goal is to use the bus as much as it's already done today.

[fredlcore]

Hm, not sure if I understand what you mean by "history" - BSB-LAN itself polls only those parameters from the bus which are defined as log_parameters (up to 20) with the interval of log_intervall seconds. You can specify another 20 parameters as avg_parameters which will be polled every minute. For these parameters a 24 hour average will be calculated. But other than that it's completely up to the home automation system to decide how many parameter it polls and in which frequency. So I'm not sure where in this process you would link in your code?

[fredlcore]

@escoand, can you get back to my follow-up question once you have the time?

[escoand]

Sure. I wasn't aware of how BSB-LAN works. So my idea would end in just sending other (or additional) MQTT telegrams than today.

[henrythasler]

I'm using node-red-contrib-bsb-lan to regularly (30s) query a set of 12 values via the http-endpoint. This takes around 28s. So this is about the bandwidth you can get (25 Values/min).

To solve the bandwidth-problem, you might need a dynamic approach where values that change more often are queried more often and more static values are queried less often. This could be done with different trigger nodes in node-red (or something else) but you might soon reach it's limits as well...

[fredlcore]

@escoand: So what do you mean with "sending other (or additional) MQTT telegrams than today"? That is what our log_parameter is for, or do you mean something else? @henrythasler: It's hard to tell beforehand which parameters change (often) and which ones not (so often), especially since it depends on what kind of heater you are using. Gas modulation for example changes every second on my heater but will never change on a heat pump. And some controllers can be used in multiple settings, so it's not really feasible to define this beforehand as part of BSB-LAN

[escoand]

I meant the MQTT discovery linked in the issue description. With this (at least Home Assistant) needs no further configuration to know everything about the sensor. But for this you have to remember which discovery message was already send.

[fredlcore]

I don't use MQTT nor do I use Home Assistant. So if you want MQTT discovery implemented, you would have to provide more information (or even better provide a PR with the necessary changes). Just like that it's not possible nor feasible for me to make any changes.

[todor-dk]

@fredlcore: Unrelated to the MQTT issue, but which parameter indicates the "gas modulation" on your boiler? If I remember it is the LM75. In those sky-high gas-price days, I'd like to see if mine is burning gas faster than the Titanic took water in.

[fredlcore]

It's 8323 (fan rpm) and 8326 (burner modulation). I think the former is a bit more accurate because sometimes the modulation is at 0% but the burner is still running...

[parsley]

Yes, 0% burner modulation is often defined at minimum burner capacity. E.g. in OpenTherm Protocol ID14 and ID17 have this kind of definition. Minimum burner capacity is of cause the lowest possible burn rate. Furthermore since the minimum capacity depends on certain circumstances you might even recognize that 0% burner modulation is not always at the same RPM rate. Therefore the RPM is a much better indicator for gas consumption. But at least in modern boilers you also find one switching and one modulating gas valve to be able to adjust the gas to air ratio (lambda control). So the RPM is a much better indicator than the boiler modulation but for a really accurate gas consumption you would also have to take the gas valve modulation value into account. In my case ist might be “2702 Sitherm Pro - Position Schrittmotor” but that’s just a guess. In fact if you just use the RPM value, you should gain a real good estimate. Perhaps combine the RPM with the gas valve information since on pre-purge and post-purge the RPM rate might be quite high but since the gas valve is closed there is no gas consumption in these phases.

Am 22.09.2022 um 23:16 schrieb fredlcore @.***>:

 It's 8323 (fan rpm) and 8326 (burner modulation). I think the former is a bit more accurate because sometimes the modulation is at 0% but the burner is still running...

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[fredlcore]

Yes, you are right, the problem with RPM, however, is that it's not linear. I tried to make an Excel sheet with the rpm values in relation to gas consumption, but couldn't find out a proper equation. For a rough estimate it is fine, though.

[todor-dk]

Interesting about the fan RPM. I will have to check the correlation between the flame indicator on the display and the 8326 burner modulation (relative output) and the 8323 fan speed. I can't say for sure, but I think on mine LM74, when 8326 is 0%, the flame indicator is off. The values for 8326 that I've seen are 0%, 2%, 10%, 20% ... 100%.

Does anybody know the correlation between 8323 and 8324 (named Current fan speed on mine)?

[fredlcore]

I'd say we move this to a separate discussion, because it has nothing to do with the original issue.

[1coderookie]

@todor-dk I remember that I had a graphic about the fan speed/power correlation etc somewhere, but I can't find it right now - maybe send me an email so I don't forget to look further and that I can send it to you directly if I'll find it..

[fredlcore]

@escoand: After we have diverted from the original issue, can you please clarify what exactly you want the code to do? Because as me and others have pointed out, publishing "all" parameters is not possible if it means querying them via the bus from the heater.

[mirkolenz]

I did not open the thread, but one major feature request in the initial post was to support MQTT autodiscovery (e.g., supported by Home Assistant). Should we open a separate issue for this functionality?

Another nice addition would be some sort of "push" updates for certain parameters. For instance, I configured a button to enable/disable the warm water in my boiler. I do not need updates for this parameter every minute, but would like to be informed if it is changed from the main console. My current workaround is that I added an automation that queries this parameter every night to synchronize it in case it has been changed on the boiler itself. It would be nice to have such functionality built into bsblan itself. But this could also be considered a separate feature request.

[fredlcore]

We can stick with the autodiscovery here because I think the functionality to constantly publish all values to BSB-LAN is not doable (at least as I understood it above). But I don't know much about this feature, so I'd need more info of what is expected exactly.

As for pushing certain parameters, this is what the logging function in combination with MQTT already does, isn't it? In any case, BSB-LAN has no further knowledge about the current state than you have. I.e. it would have to constantly/regularly have to query certain parameters and then act upon them, the same way as you do now already.

[mirkolenz]

I am on the go currently, but I will provide you with more details about this autodiscovery later.

Regarding the second topic: I also use the logging feature, but I set it to an interval of 60 seconds. I have quite a number of control values like the heating mode, comfort temp and so on. All of these combined with my regular logging parameters (like inlet temperature and modulation) are way more than twenty. However, these control values seldomly change and thus I would like to have a different interval for them. So for me the best solution would be to have a new list of "control values" in addition to the log values with a different interval. Does that make sense to you?

[fredlcore]

Yes, it makes sense, but not as a function in BSB-LAN. Setting this up would become quite tedious in the webinterface. And maybe you are content with two different log intervals, but others want three or four - that is something that should be done in the home automation software because it is technically going to be the same: In both cases, a query is sent to the bus and the data returned to the caller. Doing it via BSB-LAN would only add another middle-man. Or am I missing something else here?

[mirkolenz]

Yeah, that all makes sense. In the end, it would only be a convenience function that may not satisfy every need nonetheless. So I will just continue to handle it like I do now. As I said I will provide more information on the autodiscovery as soon as possible.

[fredlcore]

Ok, thanks

[mirkolenz]

The basic information about MQTT discovery can be found in the Home Assistant documentation. For it to work, it requires that each entity is made available under the following MQTT topic scheme: <discovery_prefix>/<component>/[<node_id>/]<object_id>/config. The discovery prefix should be configurable by the user (in most cases however, it will be homeassistant). Component denotes the type of an entity, for instance binary_sensor, switch, or sensor. The object id should be some kind of unique identifier (it does not have to be human-readable). The config then includes the regular configuration object that Home Assistant uses to set up the configured component type. This object includes (among others) the name, MQTT topics, value templates to correctly set the sensor values, units and so on.

The main question is whether this feature (if possible at all) should expose all log parameters as sensors or binary sensors only or if there should be a way to control them as well (e.g., with a switch or select field). In a best case scenario, all parameters that are writable by BSB-LAN should also be exposed with control features.

I am not in a position to know whether this feature can easily be added to the existing codebase, but I hope to have conveyed the main idea: Making available all log parameters of BSB-LAN in any home automation software that implements this automatic discovery with zero configuration necessary for the user.

[1coderookie]

Sidenote: If I'm not completely wrong, @liudger asked for something like that in the past to have BSB-LAN autodetected in HomeAssistant for his implementation. I'm not talking about the whole log parameters though, I think it was just about some kind of 'here-I-am-message' or so (iirc)..

[liudger]

Sidenote:

If I'm not completely wrong, @liudger asked for something like that in the past to have BSB-LAN autodetected in HomeAssistant for his implementation. I'm not talking about the whole log parameters though, I think it was just about some kind of 'here-I-am-message' or so (iirc)..

Hi @1coderookie ,

It's a bit different. For a integration to discover bsblan, the already implemented mdns should work. I hadn't got time to implement it yet, but will do it soon. For mqtt it's different as described above.

[fredlcore]

Since I don't use neither HA nor MQTT, could you clearly state what needs to be changed/added to the way it is being dealt with now? Please keep in mind that depending on whether people use the device-specific parameter list or the previously used "one size fits all" parameter list, BSB-LAN has no way of knowing what parameters the heater really supports. One could argue that this is the user's problem/choice and another reason to switch to the device-specitic parameter list, but I just wanted to highlight that this could be a problem. Furthermore, does this mean that actual values are also transmitted in this process or is this just the structure or available parameters and querying them is done in a different, unrelated step? If values would have to be transmitted, this would result in a full query of all parameters over the bus taking several minutes. Or are we only talking about the 20 or so log parameters that the users define?

[mirkolenz]

These are all valid questions. My proposal was concerned with making the ~20 log parameters available via the discovery feature. The following would need to be changed:

• Add config options "Enable MQTT Discovery" (bool, default: false) and "MQTT Discovery Prefix" (string, default: homeassistant).
• For all user-defined log values, publish them under the discovery prefix according to the scheme outlined above.

Conceptually, that is all there is to it. You could basically treat it as an extension to the existing MQTT logger. As of now, the configuration flow in Home Assistant for every log parameter requires you to write a custom yaml like the following:

Sensor:

unique_id: 93ee89e4-3486-478d-a2ff-20fe38d1e0d9
force_update: true
availability_topic: "bsblan/status"
state_topic: "bsblan/8310"
name: Boiler inlet temperature
value_template: "{{value | replace('---', '0.0', 1) | float}}"
unit_of_measurement: °C
device_class: temperature
icon: mdi:thermometer-chevron-up

Switch:

name: Boiler hot water
unique_id: 30d66077-1ea6-4c9a-a1a3-6871d1133525
state_topic: "bsblan/1600"
state_on: "1 - On"
state_off: "0 - Off"
command_topic: "bsblan"
payload_on: "S1600=1"
payload_off: "S1600=0"
availability_topic: "bsblan/status"
icon: mdi:water-boiler

The main question that remains is how one should deal with values that can be written to by BSB-LAN. Ideally, these would be discovered as buttons/switches/selects that allow to write to them from the home automation app. The main issue I see here is how to deal with certain options. Consider for example the following input number:

name: Boiler changeover temperature
unique_id: dac63f1c-37dd-4837-a93b-0234c600dc14
state_topic: "bsblan/730"
command_topic: "bsblan"
availability_topic: "bsblan/status"
min: 10.0
max: 20.0
step: 0.5
mode: slider
unit_of_measurement: °C
device_class: temperature
command_template: "S730={{value}}"
value_template: "{{value | float}}"
icon: mdi:sun-snowflake-variant

It is possible to define min/max/step values that control the behavior of the slider in Home Assistant. How should BSB-LAN set them correctly? If the boiler provides information about min/max values, then this would be an easy feat. But I just do not know if such information is available to you. If not, one could for instance just leave them out of the discovery payload and use the "default" values of the home automation software.

I think correctly handling this will be the main discussion topic for such a feature. Hopefully, I was able to convey the required changes better than in my last comment. If not, please let me know what is still unclear. I could also provide you with my complete configuration for various sensors/buttons and so on in Home Assistant as the discovery payload will be mostly identical to the manual configuration.

[fredlcore]

Thanks for your comprehensive feedback and sorry for the delay in getting back to it. I think I kind of understood what is necessary, and changing what text to be sent in the MQTT response should not be to big of a deal. But I don't understand two things yet:

1. How does HA know what states for example 1600 has. Is it because it would be defined as a switch which by default has on/off? But what happens with parameters such as 5960 which has several options to disaply (and set) from? Would these options have to be provided in the autoconfiguration process as well?

2. Regarding min/max values, theoretically these values can be read from more modern heaters, but it is tricky to implement because for example the minimum value for parameter 710 (comfort setpoint) is the value of 712 (reduced setpoint). And older heaters do not even provide this information which is why we decided to leave this in the hands of the user. But what would happen if no min/max values are reported? And would users still be able to add their own min/max values if they want to our would these be overwritten once the next autoconfiguration is received?

[mirkolenz]

Great to hear that this is feasible 😄 Regarding your questions:

1. Exactly: As you need to specify the domain of an entity (switch/sensor/...), HA knows how to deal with it automatically. If there are multiple options (like a mode selection), then you also need to publish all possible values via MQTT discovery. But since BSB-LAN already knows all possible values, that should not be an issue I assume.
2. I get the issue here. I think in the first iteration of this discovery feature, I would use "sensible" default values instead of correctly modeling transitive dependencies between value boundaries. I think this should not be a problem for the vast majority of users due to these two reasons:
   • If a user sends a "wrong" value to BSB-LAN (e.g., a comfort temperature lower than the reduced temperature), then BSB-LAN would behave exactly like performing a REST request: Write the value to the boiler and read the new value. The boiler would auto-correct the value and "send" it to BSB-LAN which would in turn publish it to MQTT. So a user is never shown a wrong value in HA since it would be "auto-corrected".
   • Users have the option to override all published configuration options of MQTT discovery permanently. See for example this forum post: You can customize every entity in HA regardless of the integration that created it.

[fredlcore]

Thank you for the clarifications. What I still haven't found in the linked docs is when/how often these discovery messages should be sent? Once after connecting to the MQTT broker? Probably also every time you change the log parameters?

When I look at a sample config payload like this here:

{"device_class": "temperature", "name": "Temperature", "state_topic": "homeassistant/sensor/sensorBedroom/state", "unit_of_measurement": "°C", "value_template": "{{ value_json.temperature}}" }

There are a number of things we don't know in BSB-LAN yet:

• device class: We don't actually know what kind of value a parameter has (since we have several, sometimes dozens of data types for one physical unit), but we could derive it from the physical unit that these parameters share and say: If VT_UNIT equals "°C", then use "temperature". But where is there a list of device classes supported by HA? Because we also have pressure, time and various types of gradients where I don't know if/how these would translate.
• state_topic: How would I know how this is composed? Or is it the one we later use to publish the data for a parameter?
• value_template: This seems quite HA-specific to me, not sure how to generate this out of BSB-LAN

I did not find a config which would resemble a parameter with various option values (such as 700), so not sure how this would be defined.

Since I'm no HA user, this really looks a bit beyond my scope, especially when it comes to testing this. Also, I'm not sure if this would be supporting/countering @liudger's efforts on updating the HA integration for BSB-LAN or offer a second, rather redundant approach. Or does any other home automation software uses this discovery feature?

In any case, since quite a few users are asking for using BSB-LAN with HA, I would really like to support an easy solution here. But maybe someone else takes the lead here and works together closely with me, so I can explain the innards of BSB-LAN in order to create this function for BSB-LAN without too much hassle.

[mirkolenz]

Sorry for the delay, I was quite busy this week.

• The discovery payloads are retained, so I would assume that you need to re-send them every time there is a change of configuration as well as when the retain duration expires.
• All available device classes are available in the Home Assistant Developer docs.
• You are right concerning the state_topic: It is the one that later receives the MQTT payloads.
• The value_template is indeed quite specific. But if BSB-LAN only sends numerical values without the unit as a payload, then no value template is needed. Even if it would be required for some edge cases, the syntax uses Jinja2 templates underneath, so HA uses no custom solution here.
• Parameters with multiple values can be modeled as select entities.
• I am not sure what @liudger plans for the updated integration, so I cannot comment here. I can only say that the popular zigbee2mqtt software has no HA integration, but relies only on this discovery feature. So this feature would behave exactly like a full integration, but "integrated" in BSB-LAN itself. So the risk of having a legacy integration (like it is currently) would be much reduced.

I could try to assist you in creating such a feature, but I am not sure how much I can really help. While I have quite a bit of experience with programming in Python and TypeScript, I never used C.

[fredlcore]

Ok, so what I see now is that we would need

• a function to send the discovery payloads which would be called upon boot as well as every time the configuration is changed
• a mapping between all BSB-LAN value types and their corresponding device classes in HA

As for the value template: Isn't it quite an important information to send the unit as well? For example, if we have some kind of time unit, it would be important to know whether it's seconds or minutes or hours. Furthermore, not all parameters are numeric, some are strings and some are a mix of both (for example the status parameters which have a numerical status as well as a text description). How should we deal with that?

Thanks for your offer to help, I really appreciate it and you don't have to know C for doing so. If you could write up something in Python so that I understand the general structure, that would already be a huge help. If this really would save @liudger working from adapting the official integration and also make it rather future-proof, then I would invest as much time as I can on making this work, provided that there are no inherent show-stoppers. If you think it's easier we deal with the details in direct communication via e-mail, just go ahead, otherwise we can also continue here...

[mirkolenz]

Sorry for the late reply, I am quite busy at the moment. I will contact you by mail for the following steps. If you like, we could also chat via Discord.

[fredlcore]

No hurries, take your time. E-Mail ist probably best, I have to use so many different chat services work- and activity-wise, and like none of them, so I'd be glad not to add another one ;)...

[liudger]

Hi how is this going? I am interested in helping out!

As maintaining and upgrading the homeassistant integration is not my main focus and I am the only developer. So helping another project makes more sense. My skills are mostly C# and Python and I am looking into type script. I am also learning PyTorch (maybe for optimising boiler parameters?)

Could this project help? https://homieiot.github.io Not sure if it's really relevant? (project has not many updates)

[fredlcore]

Hi @liudger, great to hear from you and that you want to help out! Since you and @mirkolenz both mentioned that C++ is not your main playground and you both know Python, this would be my suggestion: For me, it would already help a lot if you could write a program in Python that would do whatever is necessary to advertise (selected) parameters so that HomeAssistant can use these parameters with zero-configuration. I could then use this Python program and convert it to C++ and integrate it into BSB-LAN. If you think that this is doable, please use a selection of parameters in this example that covers the range of different parameter types, such as ENUM/clear-text option types, floating point types such as temperatures, on/off yes/no switches and time parameters (seconds for example).

Do you think that this is a useful approach?

[hakspiel]

Hello, if it helps, this is how i have implemented HomeAssistant AutoDiscovery in Arduino.

I have only implemented Sensors. No Switches, etc.

The function will also create a "Device" in Home Assistant automatically, and link all entities to it.

The function is called once every startup of the ESP. Once for every Sensor

Void Setup: sendMQTTDiscoveryMsg("Temperature" , "°C" , "TEMPERATURE" , "{{value_json.T}}" , String(MQTT_SENSOR_TOPIC)); sendMQTTDiscoveryMsg("Humidity" , "%" , "HUMIDITY" , "{{value_json.H}}" , String(MQTT_SENSOR_TOPIC)); sendMQTTDiscoveryMsg("DewPoint" , "°C" , "TEMPERATURE" , "{{value_json.D}}" , String(MQTT_SENSOR_TOPIC));

this is the Code of the function:

void sendMQTTDiscoveryMsg(String Sensor_kind ,  String unit ,  String Device_class ,  String Value_template ,  String State_Topic) {

/*
Example:
  // String Sensor_kind = "Temp";
  // String unit = "°C";
  // String Device_class = "temperature";
  // String Value_template = "{{ value_json.T }}";
  // String State_Topic = MQTT_SENSOR_TOPIC;  ---> MQTT Topic where the measured values of the sensor are sent to.
  // room_name =  "Office";   or Outdoor, etc

all Sensors connected to ESP:
- temperature T
- humidity H
- Pressure P
- absolute_humidity A
- lux L
- co2 C
- dew D
- Gust G
- Wind W

*/

  String room_name2 = String(room_name);
  String add_string = String("Room_Sensor");
  room_name2.toLowerCase();

  client.setBufferSize(512);   //IMPORTANT: defines the lenght of JSON message

  // This is the discovery topic for one specific sensor:
    String discoveryTopic = "homeassistant/sensor/" + String(MQTT_SENSOR_TOPIC) + "/" + Sensor_kind +"/config";

  DynamicJsonDocument doc(512);
  char buffer[512];

  doc["name"] = Sensor_kind +" " + String(room_name);    //Name of the Entity in Home Assistant

  Sensor_kind.toLowerCase();
  doc["obj_id"] = room_name2 +"_" + Sensor_kind;   //Entity ID in Home Assistant
  doc["uniq_id"] = room_name2 +"_" + Sensor_kind;   // Should be identical to obj_id
  doc["unit_of_meas"] = unit;
  if (Value_template != "") doc["val_tpl"] = Value_template;                    //Value Template; Here: JSON
  doc["stat_t"] = State_Topic;                                                    // = State Topic  
  doc["frc_upd"] = true;                                                          // Force Update
  doc["ret"] = true;                                                              //Retain needed, otherwise Home Assistant "forgets" the Sensor after Home Assistant restarts
  doc["stat_cla"] = "measurement";                                                //needed for Long term statistics  https://developers.home-assistant.io/docs/core/entity/sensor/#long-term-statistics
  if (Device_class != "") doc["dev_cla"] = Device_class;                          // Device Class needed for Long term statistics 
//  doc["state_class"] = "measurement";                                                //needed for Long term statistics  https://developers.home-assistant.io/docs/core/entity/sensor/#long-term-statistics
//  if (Device_class != "") doc["device_class"] = Device_class;                          // Device Class needed for Long term statistics 

//This part is needed to create a "Device" in Home Assistant.
//All Sensors (Entities) with the same following code are linked to the same device:

  JsonObject device = doc.createNestedObject("device");                          // create substructure in JSON
  device["name"] = add_string +"_" + String(room_name);                                                // define Device name where entity is part of it
  device["sa"] = add_string +"_" + room_name2;                                                  // define Device name where entity is part of it
  device["mdl"] = add_string +"_" + String("DIY");                                                  // define Device name where entity is part of it    
  device["mf"] = add_string +"_" + String("DIY");                                                  // define Device name where entity is part of it   
  device["sw"] = add_string +"_" + String("1.0");                                                  // define Device name where entity is part of it   
  device["ids"][0] = add_string +"_" + room_name2;                                                  // define Device name where entity is part of it

  serializeJson(doc, buffer);

  client.publish(discoveryTopic.c_str(), buffer, true);             //This Message needs to RETAIN too

delay(500);

}

[hakspiel]

This is how it appears in Home Assistant:

[fredlcore]

Oh, this looks cool! Just for my better understanding of both MQTT and HA: Would BSB-LAN still have to transmit all those parameters at a given interval or would HA poll these parameters? Because that would determine how many parameters we could advertise. For the current array that keeps the log parameters, we are limited to I think 20 or so. But if HA would poll the desired parameters at its own interval, we could just advertise all the parameters that the heating system knows (which might be several hundred).

As for the advertising during setup, could you elaborate further how to fill the parameters in the function: sendMQTTDiscoveryMsg("Temperature" , "°C" , "TEMPERATURE" , "{{value_json.T}}" , String(MQTT_SENSOR_TOPIC)); The first parameter is said to be the kind of device. How does this differ from the third parameter (device class) and what kind of options are available here? The second parameter seems to be the unit (no problem here). For the third parameter (device class) we probably would have to do a mapping such as VT_TEMP -> TEMPERATURE, VT_TEMP_WORD -> TEMPERATURE etc. Is the fourth parameter ("Temperature") the name of the parameter? Then it would probably be best if this text is generated from the category name and the parameter name, possibly prefixed with the parameter number, such as: "710 - Heat Circiut 1 - Comfort Temperature Setpoint"

How difficult would it be to add switches and dropdown selections in this scenario?

[hakspiel]

Hello, BSB-Lan still has to transmit the MQTT messages. AutoDiscovery is only an automatic Setup for HA. Autodiscovery only explains to HA how the received MQTT messages should be handled.

you can decide how to transfer MQTT messages:

JSON: PRO: in JSON you can put Data of many Sensors into 1 transfered message CON: to create a JSON Message you need memory, defined in this line: client.setBufferSize(512); //IMPORTANT: defines the lenght of JSON message

OR Single transfer: You can transfer each sensor signal separately (like BSB-Lan does). PRO: slithly easier to create and to parse CON: A lot of transfers needed, especially when you have many sensors.

I use both.

This is how my JSON messages look in HA:

This is how the Messages from BSB-LAN looks like:

This is the more detailed explanation:

Example: // String Sensor_kind = "Temp"; --> Here you are complete free to name the sensor // String unit = "°C"; --> The unit has to fit to the "Device Class" // String Device_class = "temperature"; --> see: https://developers.home-assistant.io/docs/core/entity/sensor // String Value_template = "{{ value_json.T }}"; ---> Here you describe how to handle the received data. in this example HA reads in a JSON document the Value next to "T" according to the JSON picture above it would be: 62.53

If you want to parse single messages like from the BSB_LAN example above, you can leave this blank, so: Value_template = "". If "" is received, this line ignores it: if (Value_template != "") doc["val_tpl"] = Value_template;

// String State_Topic = MQTT_SENSOR_TOPIC;

MQTT_SENSOR_TOPIC can also be free defined. I use it as part of the MQTT Folder where Home Assistant can find the MQTT data.

The example in the JSON Picture above shows "S17" so this is my Sensor No 17. In this Example MQTT_SENSOR_TOPIC = S17

This is the Code that i use to define MQTT_SENSOR_TOPIC.

    Client_Number = "1"
    MQTT_BASE_TOPIC  = "S" ;
    mqtt_topic = String(MQTT_BASE_TOPIC) + String(Client_Number);
    MQTT_SENSOR_TOPIC= mqtt_topic.c_str();

I do this because i use the exact same Sourcecode for all of my Sensors distributed in my house, and outdoor. Only thing i change is the Client_number. And based on the client number, also the "Room_name" is defined automatically:

// room_name = "Office"; or Outdoor, Sleep; Kids; etc

Auto Discovery: https://www.home-assistant.io/integrations/mqtt/#discovery-messages

To set up a swich ishould be also very easy, you only have to look tell HA where to listen, and what is a "High" signal, and what is a low signal.

like mirkolenz wrote, i also define for sensors more or less the same:

Sensor:

unique_id: 93ee89e4-3486-478d-a2ff-20fe38d1e0d9 force_update: true availability_topic: "bsblan/status" state_topic: "bsblan/8310" name: Boiler inlet temperature value_template: "{{value | replace('---', '0.0', 1) | float}}" unit_of_measurement: °C device_class: temperature icon: mdi:thermometer-chevron-up

Switch:

name: Boiler hot water unique_id: 30d66077-1ea6-4c9a-a1a3-6871d1133525 state_topic: "bsblan/1600" state_on: "1 - On" state_off: "0 - Off" command_topic: "bsblan" payload_on: "S1600=1" payload_off: "S1600=0" availability_topic: "bsblan/status" icon: mdi:water-boiler

Imporant difference for the Switch is to explain HA what message should be handled as a "High" state, in this example: state_on: "1 - On"

And where HA has to listen to it, in this example HA is listening to this topic: state_topic: "bsblan/1600"

And of course you have to explain HA how to switch the switch, in this example: payload_on: "S1600=1"

And explain HA in which MQTT topic BSB_LAN is expecting the command, here: command_topic: "bsblan"

Here is described what HA expects as minimum for a Switch: https://www.home-assistant.io/integrations/switch.mqtt/

[fredlcore]

Thanks a lot for the detailed explanation. However, the more I begin to understand the system, the more I think that the MQTT autodiscovery is maybe not the all-encompassing solution I had hoped it to be. Because if BSB-LAN still has to send the respective parameter data regularly via MQTT, this only really makes sense for parameters with changing values, such das temperatures etc. But what many people want to do (as well) is to control their heating system, i.e. set temperature setpoints or heating modes etc. But you hardly want to log for example the heating mode just so that you get this parameter with zero-configuration into HA. Do you see what I mean?

So this made me think of something else: If I could write a little script that would take BSB_LAN_custom_defs.h and generate a yaml configuration that contains all available parameters to be used in HA, would that make more sense? Then you would just add those parameters as logging parameters to BSB-LAN which you really want to monitor - and not all those that you want to be displayed in HA. These logging parameters would then need no autoconfiguration becasue the yaml configuration would already contain them.

This is just based on what I think I understood from this subject as a non-HA-user, but does it make sense, @hakspiel, @mirkolenz and @liudger?

[liudger]

I had the same idea about mqtt function. For sensors it's ideal. For the other parameters it's more handy if homeassistant trigger the refresh. I had the idea to include a mqtt function into the integration for just the sensors. But my time is currently very limited.

[hakspiel]

Hello, you understood it right, Autodiscovery makes the setup in HA easier. You have to transfer the messages still via MQTT.

I am also not sure if it is the right way to just pass over hundreds of values to HA automatically. Could be overkill.

The script is a good idea to make it easy to create the entities of interest. As i understand, i would only copy/paste the code for my "entitie of interest" into HA configuration.yaml. This is very useful for Switches/sending parameters.

For sensors where i only receive parameters, i like the current used way to define a number of sensors that i really need. Here it would be perfect to have them automatically in HA (via Autodiscovery). Just select the values in BSB_LAN UserInterface, and voila, directly available in HA.

But of course, also for sensors it would be a very useful support for BSB_LAN newbies, when the Sensors Setup Code is also availabel as Configuration.yaml example.

When i setup BSB_lan the first time most work was:

1. to understand what Values do i really need
2. How are the MQTT messages transfered
3. set it all up in HA

To sent parameters, i did not yet setup any entitie, but i would like to. What stopped me from doing this was the complex process of debugging. For Sensors it is far easier, you can see directly if it worked, and you do not need any frears that you crached your heating system.

For example playing with the offset values of the tempature sensors (outside sensor, Warmwater Temp & buffer temperature) of the heating system.

I did this in the past (before BSB_LAN) with a Hardware solution, i just added a resistor parallel to the NTC's to create the offset. Together with a manual switch. Works fine, but the offset is constant for a small temperature range only.

[mirkolenz]

My proposal would be to have a two-stage development with the following milestones:

1. Expose all existing logging parameters via MQTT discovery. For that to work, we only need to find a generic way of generating the required discovery payload.
2. Add a new control_parameters in addition to log_parameters where it is possible to specify for example the heating mode. These values would receive less updates like the log_parameters (like once every 6 hours). To make it more robust, BSB-LAN should add a "template automation" in its manual that users could copy over to their Home Assistant instance. Such a template could include triggers like the restart of the Home Assistant instance to query for changes to these parameters.

I know that this proposal may not be suitable for everyone, but I think it is a good tradeoff that at the same time does not hinder/harden future developments. The most important aspect is that a user would not have to create a custom MQTT entity for each parameter.

What do you think @liudger @fredlcore?

[liudger]

This could be a solution, but what if you manual change a control parameter (from thermostat) than when it is only refreshed every 6 hours you can't see the change? Or will this trigger you talked about used by homeassistant to refresh this data when homeassistant needs to?

Still if these are a lot of parameters then they are triggered all at once? I guess you can't just do a trigger on a few specific parameters? With a http request you can manage just the parameters you need. I am not sure yet what the best way to get this working.

I like the idea that bsblan take care of all these stuff 😄

[mirkolenz]

You are completely right, that is currently an issue: If you change a parameter on the boiler, you will not see it reflects in HASS for some time (depending on the update interval). However, I think that there is no real way to avoid this. To properly solve it, BSB-LAN would need to have "push update support" for boilers (i.e., being informed as soon as a parameter is changed internally). And if there would be support for that feature sometime in the future, BSB-LAN could just publish all received changes to MQTT which would solve this issue very elegantly.

Regarding your question on the update procedure: The parameters would not need to be updated/triggered all at once. It is even the case that the current MQTT implementation in BSB-LAN only supports querying one parameter at a time (at least to my knowledge). So you could easily add a pause between individual requests to not overwhelm the boiler's bus.

To expand a bit on this idea: I have somewhat implemented my proposal in my current HASS setup. The only thing missing is the auto-discovery (obviously), instead I needed to specify the MQTT entities manually. All control parameters like heating mode are not listed in the log_parameters, so they are not updated automatically at all. Instead, I have the following HASS automation in place:

id: ac6e5103-a68f-4719-a5d5-6e88aad911f1
description: ""
mode: single
alias: BSB-LAN
variables:
  params:
    - 700
    - 710
    - 712
    - 730
    - 1600
trigger:
  - platform: state
    entity_id: input_button.boiler_refresh_parameters
  - platform: homeassistant
    event: start
  - platform: event
    event_type: event_mqtt_reloaded
  - platform: time
    at: 03:00:00
condition: []
action:
  - repeat:
      count: "{{ params | count }}"
      sequence:
        - service: mqtt.publish
          data:
            payload: "{{ params[repeat.index - 1] }}"
            topic: bsblan
        - delay:
            hours: 0
            minutes: 0
            seconds: 2
            milliseconds: 0

What it does is every night at 3am as well after restarts of HASS or the MQTT integration, it queries the specified control parameters. In addition, I added a button to my dashboard to trigger this automation as well. Since I very rarely change settings on the boiler itself, this works perfectly for me. And even if I did, I could just click a button to see the new value reflected in HASS. If another person updates the config, it gets synced every night.

I hope my answer is somewhat easy to understand! If not, please let me know where I should provide more details 😄

[fredlcore]

Thanks all of you, I'm still not really sure which way would be the best in terms of usability and at the same time not clutter the code too much. For one, it won't be possible to do a "push update" functionality because also BSB-LAN does not know if a parameter has changed on the heater or the room unit. Theoretically, it might be possible to monitor bus messages to check if the room unit issues a SET telegram and push that parameter via MQTT. But it won't be possible on LPB-connected devices because they don't see on the bus what the room unit does. And even on BSB, it could happen that the telegram is sent over the bus while BSB-LAN is doing something else for a split of a second. So rather than offering an unreliable solution, I would rather not offer it at all.

To get this process moving, I would suggest we try to find a way to do the auto-discovery first and then find a reasonable solution for the update process. I wouldn't put too much HA-specific code into BSB-LAN, so if a polling mechanism for non-(frequently-)changing parameters could be implemented in HA directly, this would be preferrable compared to adding another pushing interval into BSB-LAN.

And maybe we just start with the easier things first: Apparently simple values and switches are easier to do than multi-selectable parameter options? We could still send the latter ones as plain text only - which would be enough anyway for read-only parameters such as the various status parameters.

As I mentioned before, my time to work on this is limited, so if someone could prepare the necessary text templates that need to be sent via MQTT during startup, I would happily add them, filled with the necessary data for each of these parameters. I also would need help to do the matching between BSB-LAN's various VT_ data types and the sensor types in HA.