Solar inverter shows a spike of negative "production"

[spitfire]

The problem

For some reason a massive negative(!) “production” bar appeared on my energy dashboard:

Meanwhile entities related to the production do not show any kind of strange data:

Looking at Developer tools->statistics does not show any either. Neither did querying the database directly (both the statistics and statistics_short_term), aside from this one - which returned slightly negative values (some strange rounding error?):

SELECT * FROM `statistics_short_term` 
INNER JOIN `statistics_meta` ON `statistics_meta`.`id` = `statistics_short_term`.`metadata_id` 
WHERE (`statistic_id` = 'sensor.sensor_t_series_pv1_energy'
OR `statistic_id` = 'sensor.sensor_t_series_pv2_energy')
AND sum < 0
ORDER BY start_ts DESC;

What version of Home Assistant Core has the issue?

core-2023.5.0

What was the last working version of Home Assistant Core?

core-2023.4.*

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

Not sure, either the db or Energy Dashboard

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[spitfire]

While yesterday was fine, it happened to me again today. It didn't happen before 2023.5 (and I did upgrade to .1 then .2 between those 2 occurrences)

[Drajv]

I have the same problem

[spitfire]

Turns out the some entities for individual device consumption have gone crazy yesterday (but not two days before) as well:

Looks like the energy dashboard is seriously broken in 2023.5.x

[cpuks]

I've got same problems - all consumption configured with utility meter so it should reset daily - midnight and all energy from previous day is showed again. Also values displayed in energy dashboard does not correspond with states of entity - ie fridge power consumption is twice bigger according to the graph.

[spitfire]

And it happened to me again. Seems to be happening only every 2nd day (or on even days) for some reason.

[cpuks]

I've got positive value from PV once utility meter is reset - that never happened before. Also all entities from individual power consumption are random values - I've got utlity meters set for daily/monthly/yearly consumption but energy dashboard displays completetly different values. @spitfire I've sent you email - we can check if there's nothing unusual on our end

Ok something is really fishy: 1st screenshot - fridge daily consumption 0.48 kWh consumed just by doing math / energy dashboard shows 0.36 kWh but once I click and do math I've got 0.48 kWh 2nd screenshot - fridge daily consumption based on utility meter -> 0.47 kWh which is right but energy dasboard shows 1.2 kWh?? Once I click I've got completely different value

[spitfire]

This is seriously messed up. Looks like my non-resetting energy meters for PV started resetting once every 2 days(!) after the upgrade. I haven't seen that mentioned in changelog, did I miss something?

[lits101]

I've had the same thing happen twice to my inverter since 2023.5.0, and once each to three separate smart plugs.

The smart plugs are Tuya, and the inverter is supplying its energy stats directly to HA via REST.

[cpuks]

Ok I'm pooling inverter data via MODBUS TCP and plugs are zigbee -> z2m. Downgraded to 2023.4.6 and no problems.

[spitfire]

@cpuks someone else told me a downgrade resolved that issue too for them. Definitely a 2023.5.x bug

[spitfire]

@cpuks someone else told me a downgrade resolved that issue too for them. Definitely a 2023.5.x bug

[Drajv]

I confirm, I went back to version 2023.4.6 and the problem disappeared

[cpuks]

So far none of developers are involved but some changes been made 2023.4->2023.5 and energy dashboard / reimann sum / utility meters are broken - we need to get attention from devs to look at this issue

[sybux]

Same problem for me since upgraded to 2023.5.x. for 3 times now, the solar production is not reset to zero at midnight but at 0:05 or 0:10 so there is a positive production during the night (I don't even know that the moon can produce solar energy :) ).

[VDRainer]

https://community.home-assistant.io/t/negative-values-in-energy-dashboard-since-last-update/568990

[Masinisti]

5.5.2023 updated from HA core_2023.4.6 to core_2023.5.1. That reseted four Riemann-sum helpers to zero and therefore Energy Dashboard shows negative for these energy entities. Two entities reseted next day 6.5.2023 and one more 8.5.2023.

I believe in all these cases source power entity was in "unavailable" state as very often withs various low power IoT devices and some why Riemann sum calculation gets reseted. That have not happened earlier on my hundreds of entities and thirty Riemann energy counters so seems that something odd is happening.

Updated 8.5.2023 to core 2023.5.2 and no new resets after update. I will report if that is happening again and trying to find solution how to set counters back to correct figures.

[1mfaasj]

same issue here today in energy dashboard: running: 2023.5.1

[Galapalos]

same kind of problem here. after update to 2023.5

i have couple tuya power sockets with riehman. one in tv stand and one in work table. Tv stand has some consumption all day 24/7 and it works with no problems, but work table has consumption only at work hours. Riehman does not work any more if there is no consumption to sum. It seem seems like if source entity is long time in zero value, riehman goes unkown/unavailable mode.

Up is kWh riehman entity and down is tuya power entity

[rrooggiieerr]

@dgomes I think you're the code owner of the Riemann sum integration?

[dgomes]

Please wait for 2023.5.3 there is fix in there that should fixe this

[nicknol]

thanks @dgomes

all see: https://github.com/home-assistant/core/issues/92392 (issue reported first time) and: https://github.com/home-assistant/core/pull/92825 (Pull request to fix the issue)

[Thrasher2020]

Please wait for 2023.5.3 there is fix in there that should fixe this

It hasn't resolved it for me (2023.5.3) :(

[MelleD]

Installed also the new version. Had a reset today and alle values to zero again :(

[spitfire]

Same here. @dgomes looks like that's an issue different from the one you've resolved?

[dgomes]

There might be more then 1 issue in this thread... as @nicknol points out, what was fixed was the error message:

could not restore last state unknown: [<class 'decimal.ConversionSyntax'>]

Now, is @spitfire error related to a Rienman Integration sensor ?

[spitfire]

There might be more then 1 issue in this thread... as @nicknol points out, what was fixed was the error message:

could not restore last state unknown: [<class 'decimal.ConversionSyntax'>]

Now, is @spitfire error related to a Rienman Integration sensor ?

Yes, it does look like it. The entities behind the individual devices I have in my energy dashboard (which are in helpers - listed as riemann integration). Seem to have started resetting every two days (!) for some reason, while they were increment ING and never resetting before:

[dgomes]

Could you please enable debug logging:

logger:
  default: warning 
  logs:
    homeassistant.components.integration: debug

[spitfire]

Could you please enable debug logging:

logger:

  default: warning 

  logs:

    homeassistant.components.integration: debug

Sure. Enabled it now. Will wait for this to reoccur.

[sybux]

Updated yesterday to 2023.5.3 (before midnight), but today, my solar production have been reset to zero once again at 0:10. Once again tonight. Before release .3, the problem was occuring every 3-4 days, now it's everyday

[dgomes]

I can't replicate the reset of the Riemann Integration sensor... there must be something more to it

Not ruling out that it is related to the fact that the Riemann integration will now follow the availability of the source sensor.

[dgomes]

What integration method are you using ?

[gieljnssns]

I can't replicate the reset of the Riemann Integration sensor... there must be something more to it

I use a template source sensor for

- platform: integration
  source: sensor.net_afname
  name: Grid consumption kWh
  unique_id: 94172690-7020-4130-9268-b140b9756539
  unit_prefix: k
  round: 3

This is the source

    - name: Net afname
      unique_id: dd4adeb0-ffa4-4e52-959f-a8b50d081b4f
      unit_of_measurement: "W"
      device_class: power
      state: >
        {%- if states("sensor.huidige_opbrengst") | float(default=0) < states("sensor.huidig_verbruik") | float(default=0) %}
            {{ (states("sensor.huidig_verbruik") | float(default=0) - states("sensor.huidige_opbrengst") | float(default=0)) | round(2) }}
        {%- elif states("sensor.huidige_opbrengst") | float(default=0) >= states("sensor.huidig_verbruik") | float(default=0) %}
            {{ 0 | float(default=0) }}
        {% endif %}

When I reload my templates there is a possibility this issue happens

[Masinisti]

I have updated 2023.5.3 and Riemann sum sensors get reseted only if I restart HA. No resets when keeping system up. So I try to avoid restarts. Some source sensors are shelly EM-sensors and some are template entities with shelly source sensors. Only once happened with Wiz power plug.

[cpuks]

Same here - positive production from PV at 00:05. Now energy dashboard charts all over the place. I'm going back to 2023.4.6 again ehh

[dgomes]

OK... that template @gieljnssns, can you add a default value if all IF's fail ?

[gieljnssns]

I have added some ELSE statements, reloaded my templates now is I have 7 unavailable sensors

Almost all of them are grid consumption

The template

    - name: Net afname
      unique_id: dd4adeb0-ffa4-4e52-959f-a8b50d081b4f
      unit_of_measurement: "W"
      device_class: power
      state: >
        {%- if states("sensor.huidige_opbrengst") | float(default=0) < states("sensor.huidig_verbruik") | float(default=0) %}
            {{ (states("sensor.huidig_verbruik") | float(default=0) - states("sensor.huidige_opbrengst") | float(default=0)) | round(2) }}
        {%- elif states("sensor.huidige_opbrengst") | float(default=0) >= states("sensor.huidig_verbruik") | float(default=0) %}
            {{ 0 | float(default=0) }}
        {% else  %}
            {{ 0 | float(default=0) }}
        {% endif %}

The integration sensor

- platform: integration
  source: sensor.net_afname
  name: Grid consumption kWh
  unique_id: 94172690-7020-4130-9268-b140b9756539
  unit_prefix: k
  round: 3

And the utility meters

daily_grid_consumption_new:
  source: sensor.grid_consumption_kwh
  periodically_resetting: false
  cycle: daily
  unique_id: 025a88aa-66f0-4430-9ab7-a6eb6c223182

monthly_grid_consumption:
  source: sensor.grid_consumption_kwh
  periodically_resetting: false
  cycle: monthly
  unique_id: 593e7109-2f96-47a6-a93f-c1b0a502ac12

yearly_grid_consumption:
  source: sensor.grid_consumption_kwh
  periodically_resetting: false
  cycle: yearly
  unique_id: bb9cf6f0-bae0-4639-94b3-3c0a21c2e0d4
quarter_hourly_grid_consumption:
  source: sensor.grid_consumption_kwh
  periodically_resetting: false
  cycle: quarter-hourly
  unique_id: 6098cdbb-97ac-428b-8b69-2f6bc7c072f4

The error in HA

Logger: homeassistant.helpers.event
Source: components/sensor/__init__.py:583
First occurred: May 16, 2023 at 00:00:00 (7 occurrences)
Last logged: 11:00:06

Error while processing state change for sensor.daily_solar
Error while processing state change for sensor.daily_energy
Error while processing state change for sensor.huidig_verbruik
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 579, in state
    numerical_value = int(value)
ValueError: invalid literal for int() with base 10: ''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 287, in _async_state_change_dispatcher
    hass.async_run_hass_job(job, event)
  File "/usr/src/homeassistant/homeassistant/core.py", line 607, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1133, in _refresh
    self.hass.async_run_hass_job(self._job, event, updates)
  File "/usr/src/homeassistant/homeassistant/core.py", line 607, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/helpers/template_entity.py", line 348, in _handle_results
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 585, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 649, in _async_write_ha_state
    state = self._stringify_state(available)
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 591, in _stringify_state
    if (state := self.state) is None:
  File "/usr/src/homeassistant/homeassistant/components/sensor/__init__.py", line 583, in state
    raise ValueError(
ValueError: Sensor sensor.grid_energy_consumption has device class energy, state class None unit kWh and suggested precision None thus indicating it has a numeric value; however, it has the non-numeric value:  (<class 'str'>)

The last time the template sensor was updated was when I reloaded them There is no grid consumption now.

[dgomes]

@gieljnssns what version of HA are you running ?

[gieljnssns]

System Information

version	core-2023.5.3
installation_type	Home Assistant OS
dev	false
hassio	true
docker	true
user	root
virtualenv	false
python_version	3.10.11
os_name	Linux
os_version	6.1.25
arch	x86_64
timezone	Europe/Brussels
config_dir	/config

Home Assistant Community Store

GitHub API | ok -- | -- GitHub Content | ok GitHub Web | ok GitHub API Calls Remaining | 5000 Installed Version | 1.32.1 Stage | running Available Repositories | 1355 Downloaded Repositories | 45 HACS Data | ok

Home Assistant Supervisor

host_os | Home Assistant OS 10.1 -- | -- update_channel | beta supervisor_version | supervisor-2023.04.1 agent_version | 1.5.1 docker_version | 23.0.3 disk_total | 457.7 GB disk_used | 147.5 GB healthy | true supported | true board | generic-x86-64 supervisor_api | ok version_api | ok installed_addons | Samba share (10.0.1), AdGuard Home (4.8.6), AppDaemon (0.12.2), Caddy (1.6), Duck DNS (1.15.0), ESPHome (2023.4.4), Mosquitto broker (6.2.1), Studio Code Server (5.5.7), WireGuard (0.8.0), MariaDB (2.6.1), Advanced SSH & Web Terminal (14.1.0), phpMyAdmin (0.8.5), Home Assistant Google Drive Backup (0.110.4), Frigate (Full Access) (0.12.0), Grocy (0.19.0), RTSPtoWebRTC (1.0.1), Glances (0.18.2), Piper (0.1.2), Whisper (0.2.0), EMHASS (0.3.7), Music Assistant BETA (2.0.0b30)

Dashboards

dashboards | 4 -- | -- resources | 14 views | 19 mode | storage

Recorder

oldest_recorder_run | May 11, 2023 at 18:44 -- | -- current_recorder_run | May 15, 2023 at 15:40 estimated_db_size | 2371.72 MiB database_engine | mysql database_version | 10.6.12

Solcast PV Forecast

can_reach_server | ok -- | -- remaining_requests | 12 rooftop_site_count | 2 polling_hours | 6 -> 22 and once at midnight

[gieljnssns]

Before 2023.5.x

Since the day I have installed 2023.5.x I have negative energy

[dgomes]

the X is important as an important patch was introduced in 2023.5.3

[gieljnssns]

This one is from yesterday, I was already on .3

[dgomes]

@gieljnssns please wait for https://github.com/home-assistant/core/pull/93137

[chickenbeef]

I've seen this issue on all the 2023.5 patch versions. Reverted back to 2023.4 and everything works again.

My devices are Tuya based with the current power reported under an attribute. The Riemann integration source is a template that reads the attribute. Example:

- platform: template
  sensors:
    geyser_current_power:
      unique_id: geyser_current_power
      friendly_name: "Geyser Power"
      value_template: >-
        {{ states.switch.geyser.attributes.current_consumption | round(2) }}
      unit_of_measurement: "W"

[dgomes]

I'm starting to see a pattern (sources are templates)

@chickenbeef can you change the template to:

{{ states.switch.geyser.attributes.current_consumption | float(default=0) | round(2) }}

[gieljnssns]

Not all of them This one fails almost every RESTART of HA

- platform: integration
  source: sensor.koffiezet_vermogen
  name: Koffiezet kWh
  unique_id: 40479fb2-765b-435e-80e5-16b91ed19b29
  method: left
  unit_prefix: k
  round: 3

Source is ESPHome

  - platform: hlw8012
    sel_pin:
      number: GPIO12
      inverted: true
    cf_pin: GPIO05
    cf1_pin: GPIO14
    current_resistor: ${current_res}
    voltage_divider: ${voltage_div}
    current:
      name: '${channel_1} stroom'
      internal: true
      unit_of_measurement: 'A'
      accuracy_decimals: 3
      icon: mdi:flash-outline
    voltage:
      name: '${channel_1} spanning'
      internal: true
      unit_of_measurement: 'V'
      icon: mdi:flash-outline
    power:
      name: '${channel_1} vermogen'
      id: vermogen
      unit_of_measurement: 'W'
      icon: mdi:flash-outline
      on_value_range:
        - above: ${max_power}
          then:
            - switch.turn_off: relay
            - homeassistant.service:
                service: persistent_notification.create
                data:
                  title: Message from ${upper_devicename}
                data_template:
                  message: Switch turned off because power exceeded ${max_power}W
    update_interval: 10s

[Thrasher2020]

Some of mine were Tasmota power plugs - I've since ditched the sum and I'm using the energy total in the energy dashboard instead. At least they are now working, but yes - some of my templated sums are doing the same as above.

[jzielke84]

Happens to me too but only if I update the host so there's a small gap in data collection.