Occasional large energy errors

[alexeiw123]

I've twice noticed that my bill calculations have shown either huge import or huge export volumes - both times I've only noticed this after the sensor data has been cleared from my recorder, however this is coming form the values reported by this add-on.

I'm not sure if this is helpful without any log data, is there a way I could set this up to capture these errors should it happen again?

[vk2him]

That's really odd, I've never seen this before. The first graph for 21 December doesn't make sense from a maths perspective - it says you returned an extraordinary 55kWh to the grid (purple bar colour) and at the same time solar production was around negative 0.6kWh - negative solar production means it was consuming rather than generating - to return to the grid the solar production would have needed to be higher than the purple 55kWh .

This leads me to think the issue "may" be within the Energy Dashboard because if either the Envoy was reporting wrong numbers or if this add-on was feeding wrong numbers, the maths within the Energy dashboard would add up. For example, if the Envoy or add-on were sending incorrect numbers, the first graph would have shown over 55kWh of solar production and not a bizarre negative number. The Dashboard simply adds/subtracts the numbers it is fed and they have to add else therein lays the problem.

For testing, perhaps on a daily basis you could keep an eye on the raw sensor graphs you create with this add-on via history before they clear in conjunction with the Energy Dashboard. Hopefully on the occasions the Energy Dashboard shows the huge discrepancies you'll be able to see the raw graphed data for that period from the add-on. Here's my last two day history as a reference.

[del13r]

I checked this link

https://developers.home-assistant.io/blog/2021/09/20/state_class_total/

I then checked dev tools and found my energy sensor has state class: total which indicates that the figure can increase or decrease.

this is not ideal for energy dashboard as the figure decreasing will cause negative figures as shown in your screenshots.

that is why it is important to use this logic in your power sensor to prevent the integration energy sensor ever recording a negative figure.

{{ [0, (production - consumption) ] | max }}

my suggestion to avoid this issue would be to change the state class to total_increasing.

For sensors with state_class total_increasing, a decreasing value is interpreted as the start of a new meter cycle or the replacement of the meter. It is important that the integration ensures that the value cannot erroneously decrease in the case of calculating a value from a sensor with measurement noise present. This state class is useful for gas meters, electricity meters, water meters etc.

[vk2him]

Great pickup @del13r - for further information you can change the state_class: from within customize.yaml

[alexeiw123]

negative solar production means it was consuming rather than generating - to return to the grid the solar production would have needed to be higher than the purple 55kWh .

Yep I agree - it's nonsense. It also shows a negative 'production' which should also be impossible, as my production, import and export are collected using the following templates:

      {% set value = states('sensor.mqtt_production') | int(default=0) %}
      {% if value  <= 1 -%}
        0
      {% elif is_state('sun.sun','below_horizon') %}
        0
      {%- else -%}
        {{ value }}
      {%- endif %}

{{ [0, states('sensor.mqtt_consumption') | int(default=0) - states('sensor.mqtt_production') | int(default=0) ] | max }}

{{ [0, states('sensor.mqtt_production') | int(default=0) - states('sensor.mqtt_consumption') | int(default=0) ] | max }}

For testing, perhaps on a daily basis you could keep an eye on the raw sensor graphs you create with this add-on via history before they clear in conjunction with the Energy Dashboard.

That is my plan - but at the moment I am not saving the MQTT sensor data to the recorder (perhaps this is contributing to the issue?), due to limited space on a 16GB card and pi 3B+ (I was using a cheap SSD but it died!). My HA yellow is due today though and I've got a better nvme SSD to go with it. I am saving the integrated power values (energy) though for the energy dashboard, as well as the accumulated energy meters for each tariff, so there would be something to review if I catch one.

found my energy sensor has state class: total which indicates that the figure can increase or decrease.

My production sensor is the same but my import and export sensors are 'total increasing'. I'll go back and fix this soon. Mine is likely quite similar to yours as I initially used your write up to get my energy dashboard and real time prices set up, although I've made many adjustments since. Awesome write up by the way! As you can see above, I'm using logic to filter negative results so I can't see how they've got in there. Maybe I am barking up the wrong tree with the MQTT add-on and this is indeed an energy dashboard bug.

My troubleshooting is a bit patchy as I'm away from home and accessing my HA install remotely.

[alexeiw123]

Great pickup @del13r - for further information you can change the state_class: from within customize.yaml

I found I was using utility_meter to reset my production daily for use in a dashboard view - this entity is energy with a state_class of total_increasing, so I've switched my energy dashboard to use this instead. Same source, just resets daily and different state_class.

[del13r]

Same source

I’m suggesting if the source sensor with attribute “device_class: energy” doesn’t also have “state_class: total_increasing”, it might have knock on effect to whatever you are using at the destination sensor

[alexeiw123]

I’m suggesting if the source sensor with attribute “device_class: energy” doesn’t also have “state_class: total_increasing”, it might have knock on effect to whatever you are using at the destination sensor

Yes true - I will fix this in customize.yaml too