Negative consumed battery

[MartinBlackburn]

Describe the bug Occasionally the consumed battery on the energy dashboard goes negative, as seen in this screenshot (in green) at 12:00, 14:00 and 17:00

I suspect this is due to the open API being a little unresponsive, as reported here: https://github.com/macxq/foxess-ha/issues/259

Which does seem to happen on the pv1_power sensor:

Though my battery charge/discharge sensors don't have missing data, or would indicate a negative value:

HA version: Core: 2024.7.2 Supervisor: 2024.06.2 Operating System: 12.4 Frontend: 20240710.0

foxess-ha version: v0.42

[FozzieUK]

Can you show which sensors you have selected in your dashboard for the battery energy_in and battery energy_out - and if they are Riemann sums or utility meters can you paste the sensor definition from your config.yaml please.

[MartinBlackburn]

Here are the sensors I'm using for the dashboard:

Most come directly from the integration, except pv1_sum, which is a template, I've redacted some of the details:

- platform: foxess
  username: xxx
  password: xxx
  deviceID: xxx
  deviceSN: xxx
  apiKey: xxx

- platform: integration
  source: sensor.foxess_pv1_power
  name: pv1_sum
  unit_time: h
  round: 2

[FozzieUK]

yes ok, if you are using the battery charge / discharge sensors directly in the energy dashboard and the OpenAPI is struggling around midnight (which it is doing) the sensors won't return to zero until they get a valid update which looking at your data is happening several hours after midnight, that is confusing the dashboard into believing the sensor has gone negative.

You could do it differently by feeding the battery charge/discharge power values into a Riemann sum (like you have with PV1) and then feed them into a utility meter (which auto resets at midnight), then use the utility meters in the energy dashboard as the battery energy in/out sources - that would stop any negative values - but it would affect your energy dashboard history.

The coding would look a bit like this depending on your inverter name -

  - method: left
    name: bat_charge_sum
    platform: integration
    round: 2
    source: sensor.foxess_bat_charge_power
    unit_time: h
  - method: left
    name: bat_discharge_sum
    platform: integration
    round: 2
    source: sensor.foxess_bat_discharge_power
    unit_time: h

utility_meter:
  bat_charge_daily:
    source: sensor.bat_charge_sum
    cycle: daily
  bat_discharge_daily:
    source: sensor.bat_discharge_sum
    cycle: daily

Fox will (hopefully) get their API fixed soon (the OpenAPI is a major frustration to everyone that uses HA cloud, Energy Stats and Fox's corporate clients - Fox have confirmed they know what needs to be done but it's a just question of when... 😞 )

[MartinBlackburn]

Ah thank you for this, I'll give it a try. I'm not too precious about the history, as it's got a few negative values now. Though it will be good to have it fixed going forward.

[MartinBlackburn]

Unfortunately, that didn't seem to work, I'm still getting a negative consumed battery:

The daily utility meters also seem slightly lower than the foxes' direct sensors:

The API seem more stable today, with no gaps, unlike previous days.

[FozzieUK]

Did you remember to swap out the sensors in the energy dashboard settings, and replace the sensors bat_charge and bat_discharge with the bat_charge_daily and, bat_discharge_daily ?

[MartinBlackburn]

I did, which does lose all the history as you warned.

[FozzieUK]

Intriguing, can you post the history trace for bat_discharge_daily and bat_charge_daily between 10am and 12noon - i'm wondering if a Fox timeout and the sensor going 'unknown' is triggering this behaviour.

[MartinBlackburn]

sure:

[MartinBlackburn]

here's the PV1 power over the same time:

[FozzieUK]

It's not obvious from the battery charge/discharge traces there are breaks but the pv1 sensors clearly shows some timeouts.

Worth giving this a try - it will force the utility meter to stay available even when the sensors below it are off-line Can you modify the utility_meters and add the always_available: setting under the cycle: for charge/discharge meters

    cycle: daily
    always_available: true

You will have to restart HA to make changes to the utility meters

[MartinBlackburn]

Will do - i'll post back later if I notice anything else odd

[MartinBlackburn]

Unfortunately, I'm still getting some negative values:

PV power, with some missing data:

Foxess direct sensor and the daily utility meter sensors, both charging and discharging:

There are a few errors in the logs:

If there is anything I can do to help, or any more data I can get you, please let me know.

[FozzieUK]

Hmmmm, there's only one thing left in there to try - the energy dashboard seems to do some clever tricks to show consumer battery / consumed solar, and I so guess the PV1 sensor might be affecting it as that is clearly going unknown.

My test system doesn't do this, and the only difference is that rather than use pv1_sum, I feed pv1_sum into a utility_meter and use that as the input for the energy dashboard.

You might have this utility_meter already set up, if not the code will look something like this but you can double check the source sensor name by looking at the config.

  daily_pv1_sum:
    source: sensor.pv1_sum
    cycle: daily
    always_available: true

Then edit your energy dashboard and change the solar production sensor to be sensor.daily_pv1_sum restart HA and then we wait ;)

[MartinBlackburn]

Unfortunately still no luck :-(

Settings for the energy dashboard, direct sensors for the grid, utility meters for the solar and battery:

PV power showing missing data:

Daily utility meter sensors:

[FozzieUK]

My test system doesn’t do this (I do get the same timeouts as you), but I don’t use any of the integration sensors directly in the energy dashboard and I think the loss of any one of these sensors is messing up the ‘consumed’ calculation the dashboard does.

I think the only way its going to be resolved is to add another couple of riemann sums and utility meters, or you could wait for Fox to resolve the OpenAPI performance issues - but in all honesty I think that will drag on for some time yet.

This would be the code for Riemann sums (double check i’ve used the correct names for your source sensors)

  - method: left
    name: feedin_sum
    platform: integration
    round: 2
    source: sensor.foxess_feedin_power
    unit_time: h
  - method: left
    name: grid_consumption_sum
    platform: integration
    round: 2
    source: sensor.foxess_grid_consumption_power
    unit_time: h

And these the utility_meters

  feedin_daily:
    source: sensor.feedin_sum
    cycle: daily
    always_available: true
  grid_daily:
    source: sensor.grid_consumption_sum
    cycle: daily
    always_available: true

You’ll need to restart HA again for the utility_meters to be loaded, give it an hour or two to start building stats then change your energy dashboard Electricity Grid settings with Grid Consumption set to sensor.grid_daily and Return to grid set to sensor.feedin_daily

[MartinBlackburn]

I've updated those - now we play the waiting game 😄

[FozzieUK]

i'm hoping that 'no news' is good news ;)

[MartinBlackburn]

So far so good. Happy to close this issue if you are. If you want, I can write up my config for the sensors and put it in the wiki. Hopefully it'll be useful for someone else

[FozzieUK]

great, thanks 👍

If you noticed the outage on Friday, it looks like Fox may have resolved the performance issues with the OpenAPI - ironically the old method would work now the servers aren't timing out multiple times a day - it's still early days but we're monitoring it.

I'll leave this open for now and check back in later in the week

[FozzieUK]

I've left it a few weeks and the Fox OpenAPI has settled down, no longer timing out - you can see the trace for the last 6 weeks it's a massive difference and the API is reliable again.

You could now switch back to your old 'total' sensors that come from the integration as these are direct reads from the inverter and won't be subject to integration 'rounding' you get through the Riemann sums; admittedly the differences should only be small, so you might prefer to stick with the way it is - should the OpenAPI ever time out the values will be correct.

[MartinBlackburn]

Good news - thank you for letting me know - feel free to close this now as it looks resolved