Energy Dashboard doesn't display 'consumed solar' in some scenarios

sexysaxon commented 1 year ago

Checklist

[X] I have updated to the latest available Home Assistant version.
[X] I have cleared the cache of my browser.
[X] I have tried a different browser to see if it is related to my browser.

Describe the issue you are experiencing

On some days 'consumed solar' isn't displayed on the energy distribution graphic on energy dashboard. Consumed solar will be shown in the main energy usage graphic for that day but is absent when looking at the week view. I have a battery on my system and this appears to happen only on nights where I charge the battery between 00:30 and 04:30 (I'm using Octopus Go tariff). This scenario only appears to happen when using large charges. on 4/9/22 I gave the battery a small charge and the problem didn't manifest. Similarly on 7/9/22 I only put in 1kWH and the problem didn't appear.

Describe the behavior you expected

Consumed solar to be shown correctly. Attached screen shot of a good day 7/11/2022

Steps to reproduce the issue

Charge a battery with over 1kWh between 00:30 and 04:30
...

What version of Home Assistant Core has the issue?

2022.8.4 and previous version

What was the last working version of Home Assistant Core?

No response

In which browser are you experiencing the issue with?

No response

Which operating system are you using to run this browser?

Windows 10

State of relevant entities

No response

Problem-relevant frontend configuration

No response

Javascript errors shown in your browser console/inspector

No response

Additional information

No response

Ralphy140 commented 1 year ago

I am seeing this as well, Also if you check the month and week view the last day of the time period often shows very little production/usage, in my case less than 3kWh even if when i go to the actual day it shows 40kWh of production and usage

sexysaxon commented 1 year ago

Update after more investigation: It seems that if the battery is charged with small amounts of less than approx 1kWh overnight then the distribution graphic exhibits different put probably related issues. The screen grab from 24th September shows the segment for self consumed solar okay but the energy flow dots don't show any flow between grid and battery. This seems to work okay when larger charges are used overnight.

calum-mcfarlane commented 1 year ago

I'm seeing the same issue here. HS 2022.11.1, with the Chestnut FoxESS HA integration, also u sing a system with a battery and Octopus Go.

The only day I can see 'Solar Consumed' in the 'Week' view is when I had only just got the integration going and it had not logged the battery charging that happened overnight. Every night since I've charged the battery at least 7kWh and it's not showing up in the week view.

I'm assuming there's a bug in the nuts and bolts of how the energy dashboard works and it'll get fixed in a future version. The actual logging of the data seems to work fine going on the daily stats view.

Edit: I'm seeing the same issue across browsers on Windows, Linux, and mobile apps for both Android and iOS.

kaitlinsm commented 1 year ago

I'm seeing this too.

I think it's to do with how the rollup is done. It looks like there's an implicit assumtion that you never charge your battery from the grid.

Bearing in mind allt hese calculations are only done with four sensors:

solar_produced(t) - kWh of solar energy produced in time period t
grid_consumed(t) - kWh drawn from the mains in time period t
grid_returned(t) - kWh sent to the grid in time period t
battery_charge(t) - kWh sent to the battery in time period t
battery_discharge(t) - kWh drawn from the battery in time period t

on the solar usage graph, it looks like we have two calculations done for each 1-hour slot, the stuff that's "positive" (i.e. energy used from all sources combined) and energy not used (energy sent to the grid or to the battery for later use)

// positive side
total_power_used = battery_discharge + grid_consumed + solar_produced - battery_charge

// negative side
total_power_unused = grid_returned + battery_charge

The assumption is - you only charge your battery with solar. So solar consumption: what did I use that I produced?

solar_consumed = solar_produced - total_power_unused
// or 
solar_consumed = solar_produced - grid_returned - battery_charge

If you do that in an hour, it works. For example, for this day between 16:00 and 17:00:

for that hour:

I produced a total of 0.35kWh of electricity from my solar panels
I consumed 0.21kWh from the grid
I sent 0.02 kWh to the grid
I sent 0.1 kWh to the battery
I consumed 0.3kWh from the battery

therefore

total_power_used = battery_discharge + grid_consumed + solar_produced - battery_charge - grid_returned
total_power_used = 0.3 + 0.21 + 0.35 - 0.1 - 0.02
total_power_used = 0.74

which matches consumed solar on the image:

total_consumed = solar_consumed + grid_consumed + battery_discharge
total_consumed = 0.23 + 0.21 + 0.3
total_consumed = 0.74

Where does solar consumed come from? I believe it's assumed that you only charge your battery with solar, so

solar_consumed = solar_produced - grid_returned - battery_charge
solar_consumed = 0.35 - 0.02 - 0.1 
solar_consumed = 0.23

whee! everything balances.

Ok, so we have our values and the sums work out. wonderful.

But what happens when we repeat this calculation over a day?

For that day:

I produced a total of 1.98kWh of electricity from my solar panels
I consumed 5.56kWh from the grid
I sent 0.15 kWh to the grid
I sent 1.8 kWh to the battery (1.3kWh during the night!)
I consumed 4.5kWh from the battery

plug that into the equations again:

total_power_used = battery_discharge + grid_consumed + solar_produced - battery_charge - grid_returned
total_power_used = 4.5 + 5.56 + 1.98 - 1.8 - 0.15
total_power_used = 10.09

which does add up:

total_consumed = solar_consumed + grid_consumed + battery_discharge
total_consumed = 0.03 + 5.56 + 4.5
total_consumed = 10.09

and solar consumed?

solar_consumed = solar_produced - grid_returned - battery_charge
solar_consumed = 1.98 - 0.15 - 1.8 
solar_consumed = 0.03

Oh look. Solar consumed for one day is LESS than the solar consumed in just one hour.

kaitlinsm commented 1 year ago

The issue here is aggregation.

The correct formula for solar_consumed(t) is sum(solar_consumed(h)) where h within t and solar_consumed(h) > 0, not solar_produced(t) - grid_returned(t) - battery_charge(t)

kaitlinsm commented 1 year ago

I've created a google sheet with the data for this day -> https://docs.google.com/spreadsheets/d/1BxD7RmeJglqGjI2Rz2wuj-PGNiVkq8ELNbydUS9P5xU/edit?usp=sharing

The columns are

Grid In (consumed from grid)
Grid Out (Sent to grid)
Solar (actual solar production)
Battery In (Battery charge)
Battery Out (power drawn from battery)
Solar consumed (calculated column for solar consumed)
Solar consumed(+) (calculated column =MAX(0, Solar Consumed)

The totals work out - my spreadhseet shows 0.03 solar consumed using the "wrong" formula, and the last column's total shows the actual solar energy consumed, a value that I think is more accurate.

The telling thing is that some columns have negative solar consumption - this is impossible.

calum-mcfarlane commented 1 year ago

@kaitlinsm nice work, hopefully this can be picked up and implemented into the next version of the energy dashboard.

kaitlinsm commented 1 year ago

Looks like my estimation of the calculation was right: https://github.com/home-assistant/frontend/blob/ac65882fdd722cbb6ae79427f26b5e26ce25801e/src/panels/lovelace/cards/energy/hui-energy-usage-graph-card.ts#L520

kaitlinsm commented 1 year ago

@bramkragten have you got any clues as to why we're seeing this weirdness?

kaitlinsm commented 1 year ago

looks like a similar issue in the energy distribution card - the solar consumption is zero if the total battery charge is more than solar produced: https://github.com/home-assistant/frontend/blob/ac65882fdd722cbb6ae79427f26b5e26ce25801e/src/panels/lovelace/cards/energy/hui-energy-distribution-card.ts#L153

calum-mcfarlane commented 1 year ago

@bramkragten have you got any clues as to why we're seeing this weirdness?

It certainly appears (coming from someone with minimal modern coding experience) that the dashboard was written with the expectation that a battery would only be charged from the panels and not from the battery. So presumably the fix is 'just' to re-factor the aggregation code to take this into account?

purcell-lab commented 1 year ago

Same issue as reported here https://github.com/home-assistant/frontend/issues/11947#issuecomment-1309916376

I have asked for this to be reopened.

I think there is an issue as energy dashboard issues do not get automatically assigned like issues raised against other integrations.

jprates commented 1 year ago

@purcell-lab called up to my attention that we already have several issues opened for what seems to be the same root cause:

https://github.com/home-assistant/frontend/issues/10163

https://github.com/home-assistant/frontend/issues/11947

https://github.com/home-assistant/frontend/issues/13701

https://github.com/home-assistant/core/issues/83549

The oldest of which goes back to 2021, we're almost on 2023 and we still have this issue around, displaying wrong data whenever batteries are used and don't just charge from solar and discharge to house, or so it seems.

It would be really nice to see this fixed, as some of us like me depend on HASS to integrate different systems, manufacturers, etc on one single platform, and as it is right now we can't trust the values it displays.

I would suggest all scenarios get together in just one single issue that would stay open, perhaps the oldest one, and the rest could be closed as a duplicate, but it would be great if we got someone to have a look at this.

Personally I'm not a programmer so I can't help getting my hands on it, but I am fully available to explain the use case scenarios I have and to test anything the devs need to see tested, anything I can do to help, count on me.

Cheers, -jprates

jprates commented 1 year ago

Just to add that after reading all posts on this issue, I'm happy to see someone else (@kaitlinsm) has reached the exact same conclusions as I did, so I would risk saying the bug is understood by now, but unfortunately it has not been fixed.

Anyway, not that anyone has asked me, but I'd vote for keeping this one issue opened as it is the best explained one thanks to @kaitlinsm, and close all others as duplicates.

Now if only anyone from the dev team would pick this... we'd be a happy group, wouldn't we? :)

My use case is here, hope it helps a bit more for us to be all on the "same page":

https://community.home-assistant.io/t/energy-management-in-home-assistant/326854/1423?u=jprates

jprates commented 1 year ago

I think there is an issue as energy dashboard issues do not get automatically assigned like issues raised against other integrations.

Any idea on how to fix this, i.e. who to call here to have a look?

fanfarestreak commented 1 year ago

I have got the same issue. Battery installed on 19th December and since then the energy dashboard shows no solar consumption.

jprates commented 1 year ago

Hi @frenck, terribly sorry for calling upon you on this issue, and wish you a happy new 2023 before anything else.

As one of the most knowledgeable and experienced developers on HASS, can you please let us know how can we get someone assigned to this issue that as you see dates back from a couple of years already? Are we doing something wrong in this issue or can we do something to move up this in the priority list?

Again, sorry for calling you here, but as you can see we got no traction at all so far, dating back to 2021.

Happy new year everyone.

fxstein commented 1 year ago

I can confirm this issue is still there. Whenever the charge from the grid is larger than the actual solar production, no solar self-consumption is shown.

mpredfearn commented 1 year ago

This is still an issue in 2023.3. The source of the problem seems to me to have been correctly identified by @kaitlinsm as the aggregation function that is applied does not take into account a battery being charged from / discharged to the grid. I'll add a screenshot of my system just to add another data point. Screenshot_2023-03-13_12-56-11 Here we can see that the battery was charged with ~5.5kWh during the night, and then ~1kWh from the solar. Around 1.5kWh was used directly. This is calculated correctly in each hour. But the totals for the day are incorrect in that they assume all power to the battery comes from the solar and as such the self-consumption is less than / equal to 0. Unfortunately my ts isn't good enough to attempt a fix - we seem to need @emontnemery or @bramkragten to look into it.

Ralphy140 commented 1 year ago

I have just had a battery installed and am seeing the exact behaviour where when the battery is charged from the grid my consumed solar is incorrect. I agree on the above view of the root of the issue so commenting to make sure this does not go stale

BrentVerhaegen commented 1 year ago

Also seeing the same behaior here but with a different scenario. I'm usually not charging (importing) from the grid but discharging (exporting) energy from the battery to the grid.

The self-consumed solar gauge also stays at 0, and the house consumed power 'wheel' shows that it all came from battery, whilst there was also some house consumed solar energy.

afbeelding

On top of that, the HA Energy dashboard considers the total consumed battery energy as house consumed energy and ingnores how much of that battery energy was exported. But that might be a separate issue.

sexysaxon commented 1 year ago

Another symptom of display issues on energy dashboard. On this particular day 28/4/23, self consumption of solar was significant but home graphic for solar consumption in Energy Distribution graphic is clearly too small and doesn't correlate with actual solar consumption.

This error is reflected in weekly and other views, 28/4/23 in weekly view shows insignificant solar consumption

ryanevans-dev commented 1 year ago

Here is another data point, I also charge and discharge the batteries. On 31st May in the day view you can see the consumed solar values:

In the week, month and year view though, you see no aggregation

bxm6306 commented 12 months ago

I'm chiming in to also +1 a fix for this issue. I too have a solar system with a battery that can be charged from Grid but the energy dashboard counts both the energy the battery pulled from Grid as well as the energy taken out of the battery which result in inaccuracies.

I'm thinking a potential solution would be to allow users to optionally choose a configured Battery System as the target for a Grid consumption entity and re-adjust the energy calculation formulas accordingly. Here's a mockup:

iainJersey commented 11 months ago

Add me to the list of people with the same issue. It has only just become apparent (UK 'summer') but it renders the graphs useless much of the time

no2chem commented 10 months ago

Hi all,

I recently added a battery which charges from grid+solar and then exports during peak and can confirm that this is still an issue.

I'm happy to try to fix it (it looks like no one has tried to fix this issue for awhile now?) as I figure out what is going on based on the thread. It looks like the data in HA is correct but the problem is in the presentation from the cards?

purcell-lab commented 10 months ago

'm happy to try to fix it (it looks like no one has tried to fix this issue for awhile now?) as I figure out what is going on based on the thread. It looks like the data in HA is correct but the problem is in the presentation from the cards?

Your assistance would be very welcome.

jprates commented 10 months ago

I'd suggest one needs to figure out all of the use cases in the first place, and how to interpret the data coming into HASS from all sensors to map to the identified scenario/use case.

It would not be much of a surprise to me if more sensors would be needed in order to fully map all possible flows to/from producers/consumers.

Starting by identifying all battery use scenarios would probably fit most of posted situations here I reckon.

iamnotralphy commented 10 months ago

The mapping of each flow is the hard part, I built my own graph to solve this using Apex Charts and needed to build the following calculated sensors to solve the issues.

These are the flows that you need a a minimum to make this work Solar2House Solar2Grid Solar2Battery Grid2House Grid2Battery Battery2House Batter2Grid

Screenshot 2023-08-07 at 9 06 15 am

The challenge was the conditions that were required to discover each flow, If you have a look further down the page in this repo you will see https://github.com/reptilex/tesla-style-solar-power-card AFP (Actual Power Flows) This goes a long was to solving the issue but there are requirements for it to work that are beyond just "add your sensor" and it works. I think that the maths is ok but having something that is user friendly enough and also caters for diffrent types of battery topology (AC vs DC coupled) will be a bit tricky.

no2chem commented 10 months ago

I see. It seems that perhaps different systems have different types of metering available?

Isn't it enough to have:

Solar Production Battery In/Out Grid In/Out

And then you can build all the sensors that @iamnhotralphy enumerated from that - and incidentally, that is all the information the energy panel asks for currently.

Yes, there are various configurations, if you have DC coupled solar that charges the batteries directly, you will need some way to measure solar production and battery input, but as long as you break it down to those 5 sensors, we can derive all the flows.

Do you see a use case that I'm missing? I think the issue is still as @kaitlinsm pointed out, aggregation isn't happening correctly for the daily totals, this shouldn't require any additional information that what the energy page already has. I'll look into the code sometime this evening.

karwosts commented 10 months ago

I think the issue is still as @kaitlinsm pointed out, aggregation isn't happening correctly for the daily totals, this shouldn't require any additional information that what the energy page already has. I'll look into the code sometime this evening.

I don't believe aggregation is done in the frontend, that's the job of the statistics platform in the core.

E.g. when in the yearly view, frontend don't ask for every hourly statistics datapoint and sum them up, it asks for the backend to return monthly sums for each statistic, and then it just gets one datapoint per month to display.

I can't think of any way to change the way this is aggregated without either adding new entity types to the energy integration (tracking "consumed solar" as a full entity with its own records in the statistics database), or just forcing the frontend to just ignore the stasistics aggregation options and always request all hourly statistics even when looking at week/month/year summaries, which I don't know if that will fly, seems pretty bad for performance.

iamnotralphy commented 10 months ago

Please forgive the crude drawing, This is the topology of my system Calculated As you can see i calculate the battery import and export as well as total house consumption. I measure 3 points Solar, House - gateway and Grid with CT clamps at 10s intervals.

Also to the point on statistics i also put all the kW sensors into a Hourly Riemann sum intergral sensor to produce kWh, This generate the required sensors to add to the energy dashboard, I believe that then in the back end it must do more statistics calculations to ensure fast response to time changes. I dont see why if the system was calculating the full list of sensors required it would not be able to do the same.

iamnotralphy commented 10 months ago

     - name: APC_solar_consumption
        unit_of_measurement: "W"
        state: >
          {% set solar_tp = states('sensor.solaredge_ac_power') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if export_to_batt > 0 %}
          {{ (solar_tp - export_to_batt) | round (1)}} 
          {% else %}
                {{ solar_tp }}
          {% endif %}

      - name: APC_total_house_consumption
        unit_of_measurement: "W"
        state: >         
          {% set solar_tp = states('sensor.solaredge_ac_power') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {{ (solar_tp - export_to_batt  ) | round (1)}}

      - name: APC_house_battery_consumption
        unit_of_measurement: "W"
        state: >         
          {% set power_purchase = states('sensor.energy_monitor_total_house_watts_draw') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if export_to_batt < 0 and (export_to_batt + power_purchase) < 0%}
          {{ ((export_to_batt + power_purchase ) * -1) | round (1)}} 
          {% else %}
                {{ 0 }}
          {% endif %}

  - name: APC_battery_to_grid
    unit_of_measurement: "W"
    state: >
      {% set power_sold = states('sensor.energy_monitor_total_house_watts_return') | float %}
      {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
      {% if power_sold > 0  and export_to_batt < 0 %}
      {{ ((power_sold  )) | round (2)}} 
      {% elif power_sold > 0 and export_to_batt > 0 and power_sold > export_to_batt %}
      {{ (( power_sold - export_to_batt )) | round (2)}}
      {% else %}
      {{0}}
      {% endif %}

      - name: APC_grid_to_battery
        unit_of_measurement: "W"
        state: >
          {% set power_purchase = states('sensor.energy_monitor_total_house_watts_draw') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if power_purchase > 0  and export_to_batt < 0 and (power_purchase + export_to_batt) > 0 %}
          {{ ((power_purchase + export_to_batt  )) | round (2)}} 
          {% elif power_purchase > 0 and export_to_batt > 0 %}
          {{ (( power_purchase )) | round (2)}}
          {% else %}
          {{0}}
          {% endif %}

- name: APC_solar_to_battery
        unit_of_measurement: 'W'
        state: >
          {% set power_sold = states('sensor.energy_monitor_total_house_watts_return') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if export_to_batt > 0 and export_to_batt - power_sold > 0 %}
          {{ ((export_to_batt - power_sold)) | round (1)}} 
          {% else %}
          {{ 0 }}
          {% endif %}

      - name: APC_grid_to_house
        unit_of_measurement: 'W'
        state: >
          {% set power_purchase = states('sensor.energy_monitor_total_house_watts_draw') | float %}
          {% set batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if power_purchase > 0 and power_purchase > -1*batt and batt < 0 %}
          {{ (( -1*batt  )) | round (2)}} 
          {% elif power_purchase > 0 and power_purchase < batt*-1 and batt < 0 %}
          {{ (( power_purchase  )) | round (2)}} 
          {% else %}
          {{0}}
          {% endif %}

      - name: APC_solar_to_grid
        unit_of_measurement: 'W'
        state: >
          {% set power_sold = states('sensor.energy_monitor_total_house_watts_return') | float %}
          {% set export_to_batt = states('sensor.solaredge_m1_ac_power_a') | float %}
          {% if power_sold > 0 and export_to_batt > 0 and power_sold < export_to_batt  %}
          {{ ((power_sold )) | round (2)}} 
          {% elif power_sold > 0 and export_to_batt > 0 and power_sold > export_to_batt   %}
          {{ (( export_to_batt )) | round (2)}}
          {% else %}
          {{0}}
          {% endif %}

purcell-lab commented 10 months ago

My case below:

Home Consumption = Solar Total + Battery Total + Grid Total 13.2 kWh = 50.6 + 8.2 - 45.6

But the energy dashboard shows Home as 43.1 kWh (which is also Battery Export Value, whenever battery export > actual home consumption I get these errors).

Using the formula above is how to calculate the Home total, but that doesn't give you the breakdown of how the Home was serviced (via Solar/ Battery or Grid)

Solar Production Battery In/Out Grid In/Out

To complete the mathematics you are required to have four variables, what is missing is House In.

Using the APF framework above (based on power) I am then able to calculate the required energy flows, using integral.

Now have a look on my right hand side, where I actually measure Home Consumption via APF.

Solar2House = 6.42 Batt2House = 3.26 Grid2House = 0.58

2House Total = 10.26 - the error from 13.2 is I suspect rounding errors from the integral function.

Screenshot 2023-08-08 17 05 12

iainJersey commented 10 months ago

One complexity is understanding where the battery charge is coming from, whether it's from PV, or grid. If there is 'surplus' PV and you are also boosting the battery (imminent load shedding for example) I can foresee issues. I'd also be cautious where EV's are being charged via a Henley Block (to avoid inadvertent transfers from the domestic battery)

purcell-lab commented 10 months ago

One complexity is understanding where the battery charge is coming from, whether it's from PV, or grid. If there is 'surplus' PV and you are also boosting the battery (imminent load shedding for example) I can foresee issues.

Mathematically if I have measurements (power or energy) for the four entities; house, grid, solar and battery then where the battery is being charged from is a formula and there are not any issues:

    - name: APF Solar2Batt
      device_class: power
      state_class: measurement
      unit_of_measurement: W
      state: >
        {% if states('sensor.apf_generation_entity')|int(default=0) > 0 and states('sensor.apf_battery_entity')|int(default=0) > 0 %}
          {% if states('sensor.apf_battery_entity')|int(default=0) > states('sensor.apf_grid2batt')|int(default=0) %}
            {% if states('sensor.apf_generation_entity')|int(default=0) - states('sensor.apf_solar2house')|int(default=0) > states('sensor.apf_battery_entity')|int(default=0) - states('sensor.apf_grid2batt')|int(default=0) %}
              {{ states('sensor.apf_battery_entity')|int(default=0) - states('sensor.apf_grid2batt')|int(default=0) }}
            {% else %}
              {{ states('sensor.apf_generation_entity')|int(default=0) - states('sensor.apf_solar2house')|int(default=0) - states('sensor.apf_solar2grid')|int(default=0) }}
            {% endif %}
          {% else %}
            0
          {% endif %}
        {% else %}
          0
        {% endif %}
    - name: APF Grid2Batt
      device_class: power
      state_class: measurement
      unit_of_measurement: W
      state: >
        {% if states('sensor.apf_grid_import')|int(default=0) > states('sensor.apf_real_house_load')|int(default=0) %}
          {{ states('sensor.apf_grid_import')|int(default=0) - states('sensor.apf_real_house_load')|int(default=0) }}
        {% else %}
          0
        {% endif %}

iainJersey commented 10 months ago

The issue is that Grid2Batt depends upon the grid import being higher than the house load. Solar PV 6kW House load 5kW Battery needs to be charged at (say 3kW) 1 kW will come from the solar surplus, and 2kW will be imported from the grid, however sensor.apf_grid_import will NOT be greater than the sensor.apf_real_house_load, so it will return 0 Perhaps I'm misthinking this The biggest issue I have is that the native energy dashboard treats battery charging overnight as 'negative solar', and so the rest of the day is totally distorted, showing incorrect self consumed solar (what exactly is 'combined from grid' ?)

purcell-lab commented 10 months ago

The issue is that Grid2Batt depends upon the grid import being higher than the house load. Solar PV 6kW House load 5kW Battery needs to be charged at (say 3kW) 1 kW will come from the solar surplus, and 2kW will be imported from the grid, however sensor.apf_grid_import will NOT be greater than the sensor.apf_real_house_load, so it will return 0 Perhaps I'm misthinking this

In that case solar2batt will be 3 kW and grid2bat will be zero. Solar2house will be 2kW and grid2house will be 3 kW.

Grid2house is always the priority and maximised by the solar inverter as there are no conversion losses it is AC to AC.

Everything else has conversion losses (5-10%): Grid2bat is AC to DC Solar2house is DC to AC Solar2batt is DC to AC to DC unless you have a DC coupled battery, but even with a DC coupled battery you have round trip losses on the way in and out of the battery.

iainJersey commented 10 months ago

Defining Solar2Batt as 3kW is a big assumption My SolarEdge (6kVA with 10kWp PV) feeds the house and the EV The surplus (returned to Grid) is collected by the AC coupled SunSynk 8kVA inverter and fed to the 30kWh of batteries, so in fact charging the batteries from solar implicitly goes DC-AC-AC-DC, whereas the Sunsynk collecting AC directly only has AC-DC, and is the more efficient route.

All this really does is to demonstrate that there is no single 'best' model, but I would like the overnight charging of the batteries not to be treated as negative solar!

sexysaxon commented 10 months ago

I don't believe you need a sensor for house in/out. The house can't generate any output, only the battery or the solar. I'm fairly certain you don't need house input as this is calculated from the other sensors. I have a Givenergy battery system and this calculates all values correctly and has no CT clamp or any other means to directly ascertain the house consumption so this must be possible, The fact that the house consumption is shown correctly in HA in most scenario supports this unless I'm seriously mistaken. Hope this helps.

iainJersey commented 10 months ago

I'd agree - I believe this is a more usual layout (please note, I haven't hooked up all the sensors yet, so it is imbalanced)

no2chem commented 10 months ago

You definitely need grid in/out. How else would you be able to tell what the house load is?

Imagine you have: solar_production = 1kWh battery_output = 0.5 kWh battery_input = 0 kWh

What is the house load? Well, that's totally dependent on what the grid is providing (or being returned!). If grid_out = 1.5kWh then the house load is 0, but if grid_in = 10kWh then the house load is 11.5kWh!

The actual problem I think is that having battery_output be solar|grid makes it impossible to know accurately what the solar consumption is just by adding numbers - @kaitlinsm's proposal is to integrate to capture the difference which should resolve the inaccuracies.

For example, let's say battery_in is 1 kWh, but when the sun was down that day. Solar production was 1 kWh. If we add, we get solar consumption = 1kWh, which is wrong because battery_in occurred when the sun was down.

The correct solution is to integrate (or aggregate). In other words, if the wrong formula for solar consumption is:

$solar\_consumption(t) = solar\_production(t) - grid\_out(t) - battery\_in(t)$

Then the correct formula should be: $solar\_consumption(t) = \int {solar\_production - grid\_out - battery\_in}\;dt$

Which solves the above problem.

The only problem here being that grid_out can include battery_out. Without a way to tell whether grid_out includes battery_out might be impossible without having a way to directly measure the house load. An example of where this could be a significant issue:

solar_production = 10kW house_load = 10kW battery_out = 10kW grid_out = 10kW

So, here's the question - is the solar consumption in this time slice 0% (the house is consuming the battery?) or 100% (the house is consuming the solar and exporting the battery?). Moreover, it seems that we would need to know what % of the battery was charged by solar too - and that presents the same problem - in this scenario, was the battery charged by solar, or the grid:

solar_production = 10kW house_load = 10kW battery_in = 10kW grid_in = 10kW

Perhaps this could be resolved if we designed some priority rules and kept them consistent. For instance, we could define that solar always goes to the house, and anything leftover goes into the battery. But this still leaves us with the problem of not knowing the house_load, and not being able to differentiate between battery export and solar export. Perhaps we could apply a priority rule on that as well? Battery export is prioritized, so perhaps the rule is then:

$solar\_consumption(t) = \int {solar\_production -max(grid\_out - battery\_out, 0) - battery\_in}\;dt$

so if there is any battery output, that gets taken into account first - "the house is consuming the solar and exporting the battery" in the previous example.

@karwosts - perhaps this can be done by having the frontend implicitly create (or request the user create) an integration platform sensor for solar consumption?

no2chem commented 10 months ago

Another significant consideration is time of accounting. Is solar consumed when the battery is charged? Or is it consumed when the battery which was charged by solar is consumed? I suspect for accounting reasons, the former may be simpler, and then there is no need to keep track of whether the battery was charged by solar or not.

purcell-lab commented 10 months ago

I would like the overnight charging of the batteries not to be treated as negative solar!

On that point we are most certainly agreed!

langestefan commented 10 months ago

I think the issue is still as @kaitlinsm pointed out, aggregation isn't happening correctly for the daily totals, this shouldn't require any additional information that what the energy page already has. I'll look into the code sometime this evening.

I don't believe aggregation is done in the frontend, that's the job of the statistics platform in the core.

The aggregation does seem to be part of the frontend: https://github.com/home-assistant/frontend/blob/dev/src/panels/lovelace/cards/energy/hui-energy-distribution-card.ts#L198

It should be possible to solve it there, but the code is kind of a maze.

Ben00793 commented 8 months ago

Why isn't this issue addressed by the developers? Pushing out big updates every month and no one cares about a basic function being buggy. So annoying!

langestefan commented 7 months ago

Why isn't this issue addressed by the developers? Pushing out big updates every month and no one cares about a basic function being buggy. So annoying!

Please try to remember almost all devs are volunteers. Why don't you commit a patch yourself?

Ben00793 commented 7 months ago

Hello guys, I don't have any experience in programing. So I've asked ChatGPT for help. Can someone who knows the programing language, please check if the solution provided by Chat GPT is possible fix or it is complete bullocks? Thanks!

First I copied the part of the code from the frontend where I think the consumed solar calculation happens:

if (summedData.from_battery) {
  if (summedData.to_grid) {
    const used_battery = {};
    for (const start of Object.keys(summedData.from_battery)) {
      used_battery[start] =
        (summedData.from_battery![start] || 0) -
        (battery_to_grid[start] || 0);
    }
    combinedData.used_battery = { used_battery };
  } else {
    combinedData.used_battery = { used_battery: summedData.from_battery };
  }
}

ChatGPT It looks like you've provided a code snippet written in JavaScript or a similar language. This code appears to be part of a larger program, and it seems to be calculating and updating some energy-related data based on inputs provided in the summedData object.

Here's a breakdown of what the code is doing:

Two empty objects, grid_to_battery and battery_to_grid, are initialized at the beginning. The code checks if there is data for energy sent to the grid (to_grid) or to the battery (to_battery), and if there is solar data (solar). Inside the loop over the keys of the solar data, it calculates the used_solar at each time point by subtracting the energy sent to the grid and the energy sent to the battery from the solar energy generated. If used_solar is negative, it means excess energy was generated. In this case, the code calculates how much energy should be sent from the grid to the battery (grid_to_battery) and updates the used_solar accordingly. It also updates the battery_to_grid based on the energy received from the grid and adjusts grid_to_battery if needed. If used_solar is less than zero, it is set to zero. The final result, used_solar, is added to the combinedData object. The second part of the code deals with energy from the battery (from_battery). If there is data for energy from the battery, it calculates the used_battery based on whether there is also energy sent to the grid (to_grid). If there is no energy sent to the grid, it simply assigns the used_battery to the from_battery data.

Please note that there are some assumptions about the structure of the summedData object, and certain properties are expected to be present.

If you have specific questions or if you'd like further clarification on any part of the code, feel free to ask!

BP In this code there is an error. If the battery is charged from the grid, the used_solar is not calculated correctly.

ChatGPT Thank you for pointing that out. I see the potential issue. It seems that the calculation of used_solar does not consider the case where energy is drawn from the grid to charge the battery. To address this, you need to adjust the calculation of used_solar in the first loop.

Here's the modified code:

javascript Copy code const grid_to_battery = {}; const battery_to_grid = {};

if ((summedData.to_grid || summedData.to_battery) && summedData.solar) { const used_solar = {}; for (const start of Object.keys(summedData.solar)) { const solarGeneration = summedData.solar[start] || 0; const energyToGrid = summedData.to_grid?.[start] || 0; const energyToBattery = summedData.to_battery?.[start] || 0; const energyFromGrid = summedData.from_grid?.[start] || 0;

used_solar[start] = solarGeneration - energyToGrid - energyToBattery;

if (used_solar[start] < 0) {
  if (summedData.to_battery) {
    grid_to_battery[start] = Math.min(-used_solar[start], energyFromGrid || 0);
    battery_to_grid[start] = -used_solar[start] - grid_to_battery[start];
  }
  used_solar[start] = 0;
}

} combinedData.used_solar = { used_solar }; }

// The rest of the code remains unchanged In this modification, I've explicitly calculated the energy drawn from the grid (energyFromGrid) and updated the calculation of used_solar accordingly. This should address the issue you mentioned. If you have any further questions or concerns, feel free to ask!

Ben00793 commented 7 months ago

Why did my last comment been marked off topic? I'm trying to fix the issue here.

Ralphy140 commented 7 months ago

Why did my last comment been marked off topic? I'm trying to fix the issue here.

I dont know it seems odd, If i had to guess something to do with the use of ChatGTP. It does seem that you are the only person that has made an effort to update the code, I dont know enough to know if its correct but i appreciate the effort :)

home-assistant / frontend