Home assistant throws "no attribute" warning

[mglatz]

Bug description A clear and concise description of what the bug is. Mention which EMS-ESP version you're using.

after update to HA core version "core-2021.4.0" and supervisor to version "supervisor-2021.03.9" I sudenly get lots of these warnings in home-assistant.log:

WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'heatingPump2Mod' when rendering '{{value_json.heatingPump2Mod}}' If I'm not mistaking I get one for each ems-esp sensor and since I pool them on change, I see 1000's errors. I don't think it has negative effect on UI, I see all the values updated. The warnings and their count is annoying though.

Steps to reproduce update HA to latest version

Device information

{
  "System": {
    "version": "2.2.0",
    "uptime": "000+00:18:31.299",
    "freemem": 43,
    "fragmem": 2
  },
  "Settings": {
    "enabled": "1",
    "publish_time_boiler": 0,
    "publish_time_thermostat": 0,
    "publish_time_solar": 0,
    "publish_time_mixer": 10,
    "publish_time_other": 0,
    "publish_time_sensor": 0,
    "mqtt_format": 3,
    "mqtt_qos": 0,
    "mqtt_retain": "0",
    "tx_mode": 1,
    "ems_bus_id": 11,
    "master_thermostat": 0,
    "rx_gpio": 13,
    "tx_gpio": 15,
    "dallas_gpio": 14,
    "dallas_parasite": "0",
    "led_gpio": 2,
    "hide_led": "0",
    "api_enabled": "1",
    "bool_format": 3,
    "analog_enabled": "0"
  },
  "Status": {
    "bus": "connected",
    "bus protocol": "HT3",
    "#telegrams received": 461,
    "#read requests sent": 133,
    "#write requests sent": 41,
    "#incomplete telegrams": 0,
    "#tx fails": 3,
    "rx line quality": 100,
    "tx line quality": 100,
    "#MQTT publish fails": 0,
    "#dallas sensors": 0
  },
  "Devices": [
    {
      "type": "Boiler",
      "name": "Condens 2500/Logamax/Logomatic/Cerapur Top/Greenstar/Generic HT3 (DeviceID:0x08, ProductID:95, Version:18.13)",
      "handlers": "0x10 0x11 0x14 0x15 0x16 0x18 0x19 0x1A 0x1C 0x2A 0x33 0x34 0x35 0xD1 0xE3 0xE4 0xE5 0xE6 0xE9 0xEA 0x494 0x495"
    },
    {
      "type": "Controller",
      "name": "HT3 (DeviceID:0x09, ProductID:95, Version:18.13)",
      "handlers": ""
    }
  ]
}

Additional context Also tried with version 2.2.1, with same result.

[ekivel]

Same issue here, following...

[proddy]

same here! I'm seeing for example:

Logger: homeassistant.helpers.template
Source: helpers/template.py:1335
First occurred: 6:03:07 (170762 occurrences)
Last logged: 20:52:20

Template variable warning: 'dict object' has no attribute 'wwstoragetemp2' when rendering '{{value_json.wwstoragetemp2}}'
Template variable warning: 'dict object' has no attribute 'wwsetpumppower' when rendering '{{value_json.wwsetpumppower}}'
Template variable warning: 'dict object' has no attribute 'tankmiddletemp' when rendering '{{value_json.tankmiddletemp}}'
Template variable warning: 'dict object' has no attribute 'wwstarts2' when rendering '{{value_json.wwstarts2}}'
Template variable warning: 'dict object' has no attribute 'lastcode' when rendering '{{value_json.lastcode}}'

I'll see if I can fix it.

[proddy]

Right, this is due to a change in Home Assistant where it no longer silently accepts templates referring to undefined variables. The missing attributes (like heatingPump2Mod in your example) is not present on your Boiler.

@MichaelDvP looks like we need to further implement the Device Flags also for boilers, like you did with DeviceFlags::EMS_DEVICE_FLAG_EMS on your GB125 so we only make calls to register_device_value() if we're 100% certain the Boiler can read that device parameter. It's going to be hard because to figure out.

The alternative is drop support for HA Discovery via MQTT and build a native interface into HA (https://github.com/emsesp/EMS-ESP/issues/87). I would only do this for v3 though (ESP32) as I really don't want to touch the v2 code anymore.

[ekivel]

So what is the solution for v2? Disable discovery and configure sensors manually, like it was with earlier versions?

[mglatz]

temporary solution is to disable warnings, add this to configuration.yaml

logger:
  default: warning
  logs:
    homeassistant.helpers.template: error

[proddy]

tell me which boiler you have (product id) and which attributes are giving errors, so at least we can start with filtering those out

[ekivel]

Boiler: Condens 2500/Logamax/Logomatic/Cerapur Top/Greenstar/Generic HT3

these are the attributes are giving the errors in HA: pumpMod2 retTemp switchTemp sysPress boilTemp flameCurr exhaustTemp setFlowTemp wwBufferBoilerTemperature setBurnPow upTimeControl upTimeCompHeating upTimeCompCooling upTimeCompWw heatingStarts coolingStarts wWStarts2 nrgConsTotal auxElecHeatNrgConsTotal auxElecHeatNrgConsHeating auxElecHeatNrgConsDHW nrgConsCompTotal nrgConsCompHeating nrgConsCompWw nrgConsCompCooling nrgSuppTotal_ nrgSuppHeating nrgSuppWw nrgSuppCooling maintenanceMessage wwStorageTemp1 wwStorageTemp2 wWSetPumpPower wwMixTemperature wwBufferTemperature floordry

[nagyrobi]

These are mine:

'pumpMod2'
'retTemp'
'switchTemp'
'sysPress'
'boilTemp'
'flameCurr'
'exhaustTemp'
'setFlowTemp'
'setBurnPow'
'upTimeControl'
'upTimeCompHeating'
'upTimeCompCooling'
'upTimeCompWw'
'heatingStarts'
'coolingStarts_'
'wWStarts2'
'nrgConsTotal'
'auxElecHeatNrgConsTotal_'
'auxElecHeatNrgConsHeating'
'auxElecHeatNrgConsDHW'
'nrgConsCompTotal'
'nrgConsCompHeating'
'nrgConsCompWw'
'nrgConsCompCooling'
'nrgSuppTotal_'
'nrgSuppHeating'
'nrgSuppWw'
'nrgSuppCooling'
'maintenanceMessage'
'maintenance'
'wwStorageTemp1'
'wwStorageTemp2'
'wWSetPumpPower'
'wwMixTemperature'
'wwBufferTemperature'

[mglatz]

these are mine for Condens 2500/Logamax/Logomatic/Cerapur Top/Greenstar/Generic HT3 and my boiler relly is junkers cerapur comfort. From what I see is that although the device desc is same, the missing values differ.

If detecting which entities work automatically is too difficult, you could probably fairly easily add a whitelist or a blacklist of sensors that will be passed to mqtt.

pumpMod2
outdoorTemp
switchTemp
boilTemp
flameCurr
exhaustTemp
setFlowTemp
setBurnPow
upTimeControl
upTimeCompHeating
upTimeCompCooling
upTimeCompWw
heatingStarts
coolingStarts_
wWStarts2
nrgConsTotal
auxElecHeatNrgConsTotal_
auxElecHeatNrgConsHeating
auxElecHeatNrgConsDHW
nrgConsCompTotal
nrgConsCompHeating
nrgConsCompWw
nrgConsCompCooling
nrgSuppTotal_
nrgSuppHeating
nrgSuppWw
nrgSuppCooling
maintenanceMessage
heatingPumpMod
heatingPump2Mod
mixerTemp
tankMiddleTemp
heatingPump
wwStorageTemp1
wwStorageTemp2
wWSetPumpPower
wwMixTemperature
wwBufferTemperature

[nagyrobi]

Guys, @proddy asked for product ID, not the name. Make sure you list what you have (number starting 0xAA) but I think a screenshot has all the needed info (can Ctrl+V the image directly in github).

[MichaelDvP]

The names are unique, we know the productID for a given name, but maybe the version makes a difference. Screenshot or call system info is a good idea.

A lot of the values can be sorted to different telegrams. I think the first step is to rule out some telegrams for the boilers. I think there are only a few telegram-sets like EMS1.0, EMS+/HT3, and heatingpumps.

I sorted the values to telegrams, a lot of values are telegrams 0x494, 0x495 which is a heatingpump described in #633 for productID 172. I think the ID95 described here uses the plus-telegrams 0xEx and not monitors 0x18/0x19, Can anybody test this (terminal read 8 18 etc. will responde with empty)?

Doesn't have HA-discovery have a way to suppress the warnings (mark as optional or something like that)?

Here the sorted values:

-> message 0xD1 and 0x19
outdoorTemp

-> message 0x19 and 0xE5
switchTemp
boilTemp
flameCurr
exhaustTemp
heatingPumpMod

-> message 0x1A
setFlowTemp
setBurnPow
wWSetPumpPower

-> message 0x495
upTimeControl
upTimeCompHeating
upTimeCompCooling
upTimeCompWw
heatingStarts
coolingStarts_
wWStarts2
nrgConsTotal
auxElecHeatNrgConsTotal_
auxElecHeatNrgConsHeating
auxElecHeatNrgConsDHW
nrgConsCompTotal
nrgConsCompHeating
nrgConsCompWw
nrgConsCompCooling

-> message 0x494
nrgSuppTotal_
nrgSuppHeating
nrgSuppWw
nrgSuppCooling

-> message 0x1C
maintenanceMessage

-> message 0xE3
heatingPump2Mod

-> message 0x2A
mixerTemp
tankMiddleTemp

-> message 0x18
heatingPump
wwStorageTemp1
wwStorageTemp2

-> can not find these:
pumpMod2
wwMixTemperature
wwBufferTemperature

[proddy]

For v3 (ESP32) I think we can fix this dynamically. When the device MQTT payload is built up in EMSdevice::generate_values_json() send the MQTT HA Discovery config if it hasn't already done so. This way any device value that is empty will be skipped. The nice thing about this approach is that it'll work for all device types and simplifies the code, like in the thermostat which is tricky.

But it doesn't help with v2 (ESP8266).

[proddy]

For v3 (ESP32) I think we can fix this dynamically. When the device MQTT payload is built up in EMSdevice::generate_values_json() send the MQTT HA Discovery config if it hasn't already done so. This way any device value that is empty will be skipped. The nice thing about this approach is that it'll work for all device types and simplifies the code, like in the thermostat which is tricky.

But it doesn't help with v2 (ESP8266).

forget that. its too tricky. the device render would need to know specifics about each device and how the HA templates work.

[MichaelDvP]

I think dynamically for HA is possible, but in addition, not simplifying the code in thermostat, etc. I'll make this change and do a PR to the webcallcmd-branch, than you can check with HA if it works. But it's only for v3.

[proddy]

works great, thanks Michael.

Note @mglatz @ekivel this is for the ESP32 only. If you're actively using v2 today and want to stay ahead of new features I strongly recommend you buy an ESP32 (4 euros on Ali, see here)

[ekivel]

OK, I will replace the chip with ESP32, I was not aware of this upgrade, but it looks easy. thx

[nagyrobi]

No fix for ESP8266??

[proddy]

No fix for ESP8266??

not an easy one no. It would mean a lot of extra work and since we're only 2 developers in our spare time making free software there are better things to focus on. Spend a few euros and upgrade?

[nagyrobi]

I just bought the device a few weeks ago, especially because it's so nice and compact and requires no external power. Now I have to brute-forcely destroy the nice enclosure, and draw an external power source to it... :-(

[MichaelDvP]

Now I have to brute-forcely destroy the nice enclosure, and draw an external power source to it...

• esp32 works with jack-power, no need for external power
• you can cut two pieces from the esp32-board and it fits without cutting the case
• you can stay with 8266 and
  • disable the warnings like @mglatz suggested
  • remove all (for your system) unused register_mqtt_ha_sensor from code an compile your privat version

proddy has tested the v3-datastructure with esp8266, but it takes to much ram and you can not load the webpage if ems-bus is connected. The v2 sends out the HA-registers without storing them, it's not possible to check if the topic has a value. A llttle step can be to flag the different boiler types and skip some values if this type does not have them, but some values depends on configuration, not on boiler-type (outdoortemp, returntemp, systempressure, switchtemp, etc.).

[nagyrobi]

* esp32 works with jack-power, no need for external power

Thanks, but my boiler lacks a jack connector.

[MichaelDvP]

Do you know that some boilers have the jack inside? https://bbqkees-electronics.nl/wiki/gateway/connecting-to-the-boiler.html

[nagyrobi]

Thank you, I am aware of those links. I just went down and triple-checked, no jack on my model. Only bus connector inside, nothing else available. It's a Junkers Ceraclass Excellence ZSC 28-3 MFA E 23.

[MichaelDvP]

Thats sad. I think you only have boiler and thermostat on the bus, there is a little chance that esp32 works buspowered with only a few rx-errors . (In my setup with mixer buspower on 8266 give a few rx-errors and esp32 gives much rx-errors, jackpower for both no errors.) If you dont want to cut the housing, this is the board-modification to make the MH-module fit (remove the plastic spacers): For 8266 i've started a test with flags on my repo and flagged the HT3 for the telegrams i think. But it's a very experimental version where i try to save more memory for 8266 and change keywords to keep it exchangeable with v3. here. But i think some HA-warning will stay.

[nagyrobi]

Thank you.

[proddy]

@nagyrobi as Michael mentioned, perhaps the easiest solution is to fork the project and comment out the 10-20 lines

[nagyrobi]

Which file?

[MichaelDvP]

Which file?

mainly src/boiler.cpp, search register_mqtt_ha_sensor and comment out the values you don't have. e.g. if you have no pressure sensor line 134:

// Mqtt::register_mqtt_ha_sensor(nullptr, nullptr, F_(sysPress), device_type(), "sysPress", F_(bar), nullptr);

[nagyrobi]

Sounds acceptable!

So if I have this error message, I should comment the lines containing the variables from it?

I see no pumpMod2 (at all in the repo).

[MichaelDvP]

This is renamed to heatingPump2Mod. in this commit from 02.03.

[nagyrobi]

Which branch should I use? dev or master?

[MichaelDvP]

There is actually not much difference, see here.

[nagyrobi]

Trying to build but I run into

In file included from src/emsesp.h:36:0,
                 from src/WebStatusService.cpp:20:
lib/framework/ESP8266React.h:33:21: fatal error: WWWData.h: No such file or directory

Lots of times...

Looks like pre:scripts/build_interface.py doesn't run on any of my systems (tried on 2 machines). Strangely no related error is shown during pio run.

[proddy]

First install node (which has npm). The pio run should call the esp8266 target which calls build_interface.py to build the WWWData.h file. Unless you've modified platformio.ini or have a pio_local.h file that overrides it.

[nagyrobi]

tried to run npm install and npm run build manually in the interface directory, but the run build also fails

[nagyrobi]

First install node (which has npm). The pio run should call the esp8266 target which calls build_interface.py to build the WWWData.h file. Unless you've modified platformio.ini or have a pio_local.h file that overrides it.

Didn't modify any of those. What do you mean by First install node (which has npm) ? If you mean nodejs, that's installed... it's v10.19.0.

[nagyrobi]

upgraded to v14.16.1, npm run build is OK now What I don't get is how come no error message is shown anywhere on failure to call build_interface.py ??

[nagyrobi]

Looking for commenting out the values. Many appear duplicated, like wwstoragetemp1 and wWStorageTemp1. What's the difference?

[proddy]

upgraded to v14.16.1, npm run build is OK now What I don't get is how come no error message is shown anywhere on failure to call build_interface.py ??

I'll see if I can repeat the steps and find out what is wrong. Are you on Windows, Linux or OSX?

[nagyrobi]

Linux.

[nagyrobi]

I wonder, what about disabling autodiscovery for Home Assistant, and adding only the existing values as manual sensors, wouldn't be a possible solution?

[nagyrobi]

@nagyrobi as Michael mentioned, perhaps the easiest solution is to fork the project and comment out the 10-20 lines

This is a much better approach! I've had tons of sensors which I wasn't aware that they are actually non-existent. HA is also much cleaner now!

[nagyrobi]

TIP: to remove error messages with custom build, after upload, in HA need to remove the device from MQTT integrations, and reboot EMS-ESP to re-detect it with correct list of sensors.

[proddy]

so how did you get it compile in the end?

[nagyrobi]

By running npm install and npm run build manually in the interface directory. These remain cached so no need to re-build the whole web interface afterwards, at least for the small changes of commenting out the lines.

[nagyrobi]

A possible soultion would be to simply have a checkbox at each sensor appearing in the web interface of EMS-ESP, to enable or disable it appearing in the autodiscovery for Home Assistant.

This way, not only the sensors causing such errors could be outruled from HA, but also other valid sensors could be prevented from appearing in HA if the user doesn't want to have them there.

[proddy]

yes, that's also an option. The thing is it's a ton of work and we've all moved onto to v3 now which has a different architecture

[nagyrobi]

Problem is that the ESP8266 version is not marketed as unsupported.

It is still sold as the all time bestseller and includes everything and For all EMS 1.0 boilers this product is still good choice. Me having an EMS 1.0 boiler (without jack socket) I didn't even consider an ESP32 one which requires an external power supply.

I feel a little deceived now, as I bought it just recently. IMHO the correct way to sell this should be stated that software maintenance for ESP8266 is stopped, everyone should buy at his own risk.

[proddy]

I'll keep maintaining and fixing the ESP8266 version as mentioned in the docs. I'm just not adding major new changes as there's simply no more memory left for any more code. Remember it's free open-source software so everyone is welcome to contribute. Unlike Tasmota and Home Assistant there's only 2 of us on this project, in any spare time we have.

Anyway, I thought of another workaround. Change EMS-ESP to publish all the boiler values in the MQTT payload, even if they are not present. This will stop the HA MQTT Discovery component from complaining but will give you a lot of void sensors with no values. What do you think of that?

Lastly I see from your GH you've done some work on HA configs and Python. Why not help out and take a look at getting a real interface into HA using both APIs as suggested earlier in https://github.com/emsesp/EMS-ESP/issues/756#issuecomment-816093167?

[nagyrobi]

Anyway, I thought of another workaround. Change EMS-ESP to publish all the boiler values in the MQTT payload, even if they are not present. This will stop the HA MQTT Discovery component from complaining but will give you a lot of void sensors with no values. What do you think of that?

Well from the end-user side that's how it looked until now. Had tons of sensors, to be honest I was't even aware that 80% were void anyway. I also noticed a significant performance drop after HA upgrade causing the errors, which magically disappeared after commenting out the void sensors in the firmware yesterday. Seems that processing so many errors, even though muted from the log, causes high stress on HA side. So if we get back the big sensors list that't OK but that souldn't cause any extra load on HA to process the no values.

Maybe flag them as 'unavailable'? Something like If 'no value' then 'unavailable'. Or set all of them as unavailable in LWT and become available one by one when data arrives. No problem if they are unavailable at start, it's enough to become available when first value comes. Thus any sensor who would never get data because boiler doesn't send it, would remain unavailable forever and that's correct.

Would be useful for endusers too to learn which sensors actually send data from specific boilers.