Energy doesn't support negative power prices

[OmgImAlexis]

The problem

I'm getting this error even though it should be completely valid to have a negative price.

The following entities have a negative state while a positive state is expected:
sensor.home_general_energy_price (-0.06)

I do get paid to use power during this time.

What is version of Home Assistant Core has the issue?

core-2021.9.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

No response

Link to integration documentation on our website

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

https://github.com/madpilot/hass-amber-electric/issues/24

[OmgImAlexis]

I don't know if this is related but now my energy dashboard is really broken.

This is today. As you can see this worked fine right until it hit the negative prices. At that point in values seem to be recorded incorrectly.

This is yesturday.

This is the history for the "House" sensor. Apart from the 0 it all looks fine.

This is the history for the power price.

[probot-home-assistant[bot]]

energy documentation energy source _{^{(message by IssueLinks)}}

[probot-home-assistant[bot]]

Hey there @home-assistant/core, mind taking a look at this issue as it has been labeled with an integration (energy) you are listed as a code owner for? Thanks! _{^{(message by CodeOwnersMention)}}

[emontnemery]

I think there might be two problems here:

1. The cost sensor does not handle negative prices
2. Your energy sensor has an unexpected drop to 0

Your screenshots are all about the energy sensor and the usage which goes crazy because of the drop to 0. The problem here is that the drop to 0 is interpreted as the start of a new billing cycle. Which integration is providing the energy sensor?

The cost sensor warning about negative prices is a bug in Home Assistant which needs to be fixed. Can you please share some screenshots of the cost sensor too?

[OmgImAlexis]

What do you mean by the cost sensor?

[OmgImAlexis]

If you look the 0 happened around the same time as the negative price so I have a feeling it’s related. If I’m wrong or you need more details please let me know.

[emontnemery]

The energy dashboard allows you to add both a sensor which provides the energy consumption and a a sensor which provides the energy price.

The cost sensor does not handle negative prices

The energy consumption sensor together with the price sensor will calculate the cost of the electricity consumption, and that sensor doesn't currently handle negative energy price, that's why you get this error message:

The following entities have a negative state while a positive state is expected:
sensor.home_general_energy_price (-0.06)

It would be very helpful if you could share a screenshot of the cost graph too.

Your energy usage looks strange, with a huge bump of ~100kWh around noon

This is a problem with the energy usage calculation, which is derived from sensor.house_total_consumption. Note that this is energy consumption in kWh, not cost in AUD. The reason why the energy usage graph has a huge bump is because that sensor sensor.house_total_consumption was briefly at 0, which is interpreted as the start of a new billing cycle.

In order to have some progress with this problem, can you please share details on which integration is providing sensor.house_total_consumption?

[OmgImAlexis]

Okay. Thanks for explaining that. 😃

It would be very helpful if you could share a screenshot of the cost graph too.

This?

This is sensor.house_total_consumption.

- sensor:
    - name: "House"
      unique_id: "house_total_consumption"
      unit_of_measurement: "kWh"
      device_class: energy
      state_class: total_increasing
      state: >
        {%- set sensor = states.sensor %}
        {%- set ellas_desk = sensor.ellas_desk_total_consumption.state | float %}
        {%- set lounge_room_entertainment_unit = sensor.lounge_room_entertainment_unit_total_consumption.state | float %}
        {%- set washing_machine = sensor.washing_machine_total_consumption.state | float %}
        {%- set office = sensor.office_total_consumption.state | float %}
        {%- set evies_room = sensor.evies_desk_total_consumption.state | float %}
        {%- set alexis_room = sensor.alexis_room_total_consumption.state | float %}
        {%- set sub_total_1 = ellas_desk + lounge_room_entertainment_unit + washing_machine + office %}
        {%- set sub_total_2 = evies_room + alexis_room %}
        {{ (sub_total_1 + sub_total_2) | float }}

[emontnemery]

A solution for the first problem - the cost sensor does not handle negative prices - is included in https://github.com/home-assistant/core/pull/55955, it won't land until HA 2021.10 though https://github.com/home-assistant/core/pull/55962 which should make it to next 2021.9 bugfix release.

The second problem - your energy sensor has an unexpected drop to 0 - is related to the template sensor's state being set to 0. Either there's something wrong with the template, or the underlying sensors sometimes return 0. Which integration(s) provide the sensors, sensor.ellas_desk_total_consumption, sensor.washing_machine_total_consumption and so on?

Note: In Home Assistant 2021.10, you should change the configuration of house_total_consumption to state_class: total, then it's perfectly fine if the value decreases as long as it recovers again.

[OmgImAlexis]

should make it to next 2021.9 bugfix release.

Perfect. 😃

Which integration(s) provide the sensors

They're all coming from https://www.home-assistant.io/integrations/tplink/ and they're all model KP115.

Note: In Home Assistant 2021.10, you should change the configuration of house_total_consumption to state_class: total, then it's perfectly fine if the value decreases as long as it recovers again.

No worries, thanks for the help.

[balloob]

2021.9.5 has been released with this fix included.

[emontnemery]

The problem in the issue description is fixed, but the problem with the sensor.house_total_consumption returning 0 is not.

@OmgImAlexis can you try to find what was going on around 11.45-ish when the sensor dropped? Was Home Assistant restarted? Are there 0-states in the states table for all the tplink-sensors?

[OmgImAlexis]

So I changed house_total_consumption to state_class: total_increasing and then updated to 2021.9.5 and now my house_total_consumption shows as unknown.

I don't think it was restarted. It also happened again btw. I'm pretty sure there was no restart this time either.

[OmgImAlexis]

Are there 0-states in the states table for all the tplink-sensors?

Where would I look for this?

[emontnemery]

Never mind about that, the problem is likely in the template, | float will actually return 0 if the input is not a valid number. It means if any of the sensors is unavailable, that sensor's value will be replaced by 0.

In this case it makes more sense to fail, if you update the template to use the float() function instead of the | float filter it should do just that:

{{float(states('sensor.no_such_sensor'))}} -> `unknown` this uses Home Assistant's `forgiving_float` and returns unknown
{{(states('sensor.no_such_sensor')) | float }} - > this uses jinja's float and returns 0

That is, change to:

{%- set ellas_desk = float(sensor.ellas_desk_total_consumption.state) %}

Related PR: https://github.com/home-assistant/core/pull/56051

[OmgImAlexis]

Okay updating all those lines to use float() instead of | float, I'm still seeing this in the energy settings page though.

Entity unavailable
The state of these configured entities are currently not available:
sensor.house_total_consumption (unknown)

[emontnemery]

What's the configuration of the template sensor after your changes?

[OmgImAlexis]

There's a very good chance I messed something up.

- sensor:
    - name: "House"
      unique_id: "house_total_consumption"
      unit_of_measurement: "kWh"
      device_class: energy
      state_class: total_increasing
      state: >
        {%- set sensor = states.sensor %}
        {%- set ellas_desk = float(sensor.ellas_desk_total_consumption.state) %}
        {%- set lounge_room_entertainment_unit = float(sensor.lounge_room_entertainment_unit_total_consumption.state) %}
        {%- set washing_machine = float(sensor.washing_machine_total_consumption.state) %}
        {%- set office = float(sensor.office_total_consumption.state) %}
        {%- set evies_room = float(sensor.evies_desk_total_consumption.state) %}
        {%- set alexis_room = float(sensor.alexis_room_total_consumption.state) %}
        {%- set sub_total_1 = ellas_desk + lounge_room_entertainment_unit + washing_machine + office %}
        {%- set sub_total_2 = evies_room + alexis_room %}
        {{ float(sub_total_1 + sub_total_2) }}

[OmgImAlexis]

If I put the above into the developer tools template page I get the following. So it seems the sensor is returning a number correctly?

- sensor:
    - name: "House"
      unique_id: "house_total_consumption"
      unit_of_measurement: "kWh"
      device_class: energy
      state_class: total_increasing
      state: >
        209.82000000000002

[emontnemery]

Yeah, that looks good, in the developer tools states page you can check the state of the template sensor.

You do set the unique_id to house_total_consumption, that doesn't mean your sensor's entity id will be sensor.house_total_consumption though. When a unique ID is set, the entity ID can be changed through the UI.

Also, you mentioned that:

I changed house_total_consumption to state_class: total_increasing and then updated to 2021.9.5 and now my house_total_consumption shows as unknown

What was the working configuration, wasn't the state_class already total_increasing?

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[emontnemery]

@OmgImAlexis You never replied to my last message, is this still not working for you?

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[OmgImAlexis]

Sorry I really need to clear my inbox as I never saw this reply.

Checking on my current setup I still have an issue but I have no clue if this is it or I have something else wrong.

[github-actions[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.