HA restart causes Previous day accumulation cost to be incorrect due to lack of completed dispatches

[pacemaker82]

Describe the bug

Integration receives an update for previous day's accumulation kWh and cost is calculated. With IOG this includes completed dispatches applied. Cost is accurate.

When you restart HA after this happens, the data is again retrieved from octopus and re-calculated, this time without the completed dispatches, and only based on the base tariff information. This leads to a wildly incorrect calculation.

Checking the completed dispatches, they are there in the sensor, but are not reflected as being used to calculate the previous day accumulated cost.

Reproduction steps

Restart HA

Previous day accumulation cost is calculated, without completed dispatches.

Check sensor for completed dispatches, confirm they are there.

previous day accumulation cost remains incorrect hours later.

Expected behaviour

Previous day accumulation cost reflects base tariff + completed dispatches.

Tariff Code

E-1R-INTELLI-BB-VAR-23-03-01-J

Integration Version

10.1.10

Home Assistant Version

2024.1.5

Fresh Install?

Fresh install

Home Assistant Logs

na

Confirmation

• [X] I confirm that I cannot find my solution within the documentation
• [X] I confirm that I cannot find my solution within the FAQ

[pacemaker82]

Here's the attributes for the previous day accumulation cost not taking into consideration the dispatches. Have also copied the dispatches below from theintelligent_dispatching sensor. For ease look at the 10.00-10.30 timeframe (should be cheap per completed dispatches below).

Tariff code E-1R-INTELLI-BB-VAR-23-03-01-J Charges

• start: '2024-02-12T00:00:00+00:00' end: '2024-02-12T00:30:00+00:00' rate: 0.074999 consumption: 1.58 cost: 0.12
• start: '2024-02-12T00:30:00+00:00' end: '2024-02-12T01:00:00+00:00' rate: 0.074999 consumption: 0.019 cost: 0
• start: '2024-02-12T01:00:00+00:00' end: '2024-02-12T01:30:00+00:00' rate: 0.074999 consumption: 0.004 cost: 0
• start: '2024-02-12T01:30:00+00:00' end: '2024-02-12T02:00:00+00:00' rate: 0.074999 consumption: 0.076 cost: 0.01
• start: '2024-02-12T02:00:00+00:00' end: '2024-02-12T02:30:00+00:00' rate: 0.074999 consumption: 0.007 cost: 0
• start: '2024-02-12T02:30:00+00:00' end: '2024-02-12T03:00:00+00:00' rate: 0.074999 consumption: 2.74 cost: 0.21
• start: '2024-02-12T03:00:00+00:00' end: '2024-02-12T03:30:00+00:00' rate: 0.074999 consumption: 3.163 cost: 0.24
• start: '2024-02-12T03:30:00+00:00' end: '2024-02-12T04:00:00+00:00' rate: 0.074999 consumption: 2.786 cost: 0.21
• start: '2024-02-12T04:00:00+00:00' end: '2024-02-12T04:30:00+00:00' rate: 0.074999 consumption: 0.023 cost: 0
• start: '2024-02-12T04:30:00+00:00' end: '2024-02-12T05:00:00+00:00' rate: 0.074999 consumption: 1.907 cost: 0.14
• start: '2024-02-12T05:00:00+00:00' end: '2024-02-12T05:30:00+00:00' rate: 0.074999 consumption: 0.007 cost: 0
• start: '2024-02-12T05:30:00+00:00' end: '2024-02-12T06:00:00+00:00' rate: 0.312084 consumption: 0.005 cost: 0
• start: '2024-02-12T06:00:00+00:00' end: '2024-02-12T06:30:00+00:00' rate: 0.312084 consumption: 0.006 cost: 0
• start: '2024-02-12T06:30:00+00:00' end: '2024-02-12T07:00:00+00:00' rate: 0.312084 consumption: 0.007 cost: 0
• start: '2024-02-12T07:00:00+00:00' end: '2024-02-12T07:30:00+00:00' rate: 0.312084 consumption: 0.005 cost: 0
• start: '2024-02-12T07:30:00+00:00' end: '2024-02-12T08:00:00+00:00' rate: 0.312084 consumption: 0.014 cost: 0
• start: '2024-02-12T08:00:00+00:00' end: '2024-02-12T08:30:00+00:00' rate: 0.312084 consumption: 0.011 cost: 0
• start: '2024-02-12T08:30:00+00:00' end: '2024-02-12T09:00:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T09:00:00+00:00' end: '2024-02-12T09:30:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T09:30:00+00:00' end: '2024-02-12T10:00:00+00:00' rate: 0.312084 consumption: 1.81 cost: 0.56 - start: '2024-02-12T10:00:00+00:00' end: '2024-02-12T10:30:00+00:00' rate: 0.312084 consumption: 2.133 cost: 0.67
• start: '2024-02-12T10:30:00+00:00' end: '2024-02-12T11:00:00+00:00' rate: 0.312084 consumption: 1.245 cost: 0.39
• start: '2024-02-12T11:00:00+00:00' end: '2024-02-12T11:30:00+00:00' rate: 0.312084 consumption: 1.343 cost: 0.42
• start: '2024-02-12T11:30:00+00:00' end: '2024-02-12T12:00:00+00:00' rate: 0.312084 consumption: 1.905 cost: 0.59
• start: '2024-02-12T12:00:00+00:00' end: '2024-02-12T12:30:00+00:00' rate: 0.312084 consumption: 0.017 cost: 0.01
• start: '2024-02-12T12:30:00+00:00' end: '2024-02-12T13:00:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T13:00:00+00:00' end: '2024-02-12T13:30:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T13:30:00+00:00' end: '2024-02-12T14:00:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T14:00:00+00:00' end: '2024-02-12T14:30:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T14:30:00+00:00' end: '2024-02-12T15:00:00+00:00' rate: 0.312084 consumption: 0 cost: 0
• start: '2024-02-12T15:00:00+00:00' end: '2024-02-12T15:30:00+00:00' rate: 0.312084 consumption: 0.016 cost: 0
• start: '2024-02-12T15:30:00+00:00' end: '2024-02-12T16:00:00+00:00' rate: 0.312084 consumption: 0.014 cost: 0
• start: '2024-02-12T16:00:00+00:00' end: '2024-02-12T16:30:00+00:00' rate: 0.312084 consumption: 0.009 cost: 0
• start: '2024-02-12T16:30:00+00:00' end: '2024-02-12T17:00:00+00:00' rate: 0.312084 consumption: 0.011 cost: 0
• start: '2024-02-12T17:00:00+00:00' end: '2024-02-12T17:30:00+00:00' rate: 0.312084 consumption: 0.005 cost: 0
• start: '2024-02-12T17:30:00+00:00' end: '2024-02-12T18:00:00+00:00' rate: 0.312084 consumption: 0.081 cost: 0.03
• start: '2024-02-12T18:00:00+00:00' end: '2024-02-12T18:30:00+00:00' rate: 0.312084 consumption: 0.026 cost: 0.01
• start: '2024-02-12T18:30:00+00:00' end: '2024-02-12T19:00:00+00:00' rate: 0.312084 consumption: 2.991 cost: 0.93
• start: '2024-02-12T19:00:00+00:00' end: '2024-02-12T19:30:00+00:00' rate: 0.312084 consumption: 0.022 cost: 0.01
• start: '2024-02-12T19:30:00+00:00' end: '2024-02-12T20:00:00+00:00' rate: 0.312084 consumption: 2.96 cost: 0.92
• start: '2024-02-12T20:00:00+00:00' end: '2024-02-12T20:30:00+00:00' rate: 0.312084 consumption: 3.037 cost: 0.95
• start: '2024-02-12T20:30:00+00:00' end: '2024-02-12T21:00:00+00:00' rate: 0.312084 consumption: 1.892 cost: 0.59
• start: '2024-02-12T21:00:00+00:00' end: '2024-02-12T21:30:00+00:00' rate: 0.312084 consumption: 1.39 cost: 0.43
• start: '2024-02-12T21:30:00+00:00' end: '2024-02-12T22:00:00+00:00' rate: 0.312084 consumption: 1.461 cost: 0.46
• start: '2024-02-12T22:00:00+00:00' end: '2024-02-12T22:30:00+00:00' rate: 0.312084 consumption: 1.376 cost: 0.43
• start: '2024-02-12T22:30:00+00:00' end: '2024-02-12T23:00:00+00:00' rate: 0.312084 consumption: 1.317 cost: 0.41
• start: '2024-02-12T23:00:00+00:00' end: '2024-02-12T23:30:00+00:00' rate: 0.312084 consumption: 1.296 cost: 0.4
• start: '2024-02-12T23:30:00+00:00' end: '2024-02-13T00:00:00+00:00' rate: 0.074999 consumption: 1.287 cost: 0.1

And below is the completed dispatches:

Completed dispatches - start: '2024-02-12T10:00:00+00:00' end: '2024-02-12T10:30:00+00:00' charge_in_kwh: -1.03 source: null location: AT_HOME

• start: '2024-02-12T09:30:00+00:00' end: '2024-02-12T10:00:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T11:00:00+00:00' end: '2024-02-12T11:30:00+00:00' charge_in_kwh: -0.57 source: null location: AT_HOME
• start: '2024-02-12T10:30:00+00:00' end: '2024-02-12T11:00:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T11:30:00+00:00' end: '2024-02-12T12:00:00+00:00' charge_in_kwh: -0.57 source: null location: AT_HOME
• start: '2024-02-12T18:30:00+00:00' end: '2024-02-12T19:00:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T20:00:00+00:00' end: '2024-02-12T20:30:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T19:30:00+00:00' end: '2024-02-12T20:00:00+00:00' charge_in_kwh: -0.57 source: null location: AT_HOME
• start: '2024-02-12T21:30:00+00:00' end: '2024-02-12T22:00:00+00:00' charge_in_kwh: -0.69 source: null location: AT_HOME
• start: '2024-02-12T21:00:00+00:00' end: '2024-02-12T21:30:00+00:00' charge_in_kwh: -1.03 source: null location: AT_HOME
• start: '2024-02-12T20:30:00+00:00' end: '2024-02-12T21:00:00+00:00' charge_in_kwh: -1.03 source: null location: AT_HOME
• start: '2024-02-12T22:30:00+00:00' end: '2024-02-12T23:00:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T22:00:00+00:00' end: '2024-02-12T22:30:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME
• start: '2024-02-12T23:30:00+00:00' end: '2024-02-13T00:00:00+00:00' charge_in_kwh: -1.03 source: null location: AT_HOME
• start: '2024-02-12T23:00:00+00:00' end: '2024-02-12T23:30:00+00:00' charge_in_kwh: -1.15 source: null location: AT_HOME

[BottlecapDave]

Hello and sorry that you're seeing this. I assume that the cost corrects itself eventually. I think what I'll do is delay the calculation until dispatch information is available. A fix should be available in the next release.

[pacemaker82]

Hi @BottlecapDave - no the cost hasnt corrected since the HA restart yesterday (nearly 22 hours ago now). It doesnt seem to re-evaluate. I presume it will only re-evaluate (and be correct) when new data comes in today for yesterday's data. As long as I dont restart HA it will be OK - guess there's some kind of race condition or order of events when HA reloads where the completed dispatches werent ready in time to do the calculation on restart.

As you can see above, it seems to get the data from API often but isnt re-evaluating it. 09.20am was the restart time

[pacemaker82]

Just wanted to comment that the cost "fixed" itself once the new data came in for the next previous day (if that makes sense). Meaning if the data was the same, it didnt re-evaluate and remained incorrect. The following previous day's data was correct and using the completed dispatches as expected.

[BottlecapDave]

Hello. This should hopefully be fixed in v10.1.1. Please confirm when you can :)

[pacemaker82]

This seems to have worked OK, however something interesting happened.

The previous day import cost entity updated this morning around 8.35am to £2.49. I then updated to 10.1.1, the price has changed again, now £2.10 (you can see the price changes here toward the end of the chart). I am now unsure which price is or was accurate (the pre 10.1.1 price, or the 10.1.1/10.1.2 price).

Below is the price chart and the kWh chart. You can see the total kWh didnt change, but the total price did.

[pacemaker82]

Actually, it looks like the pre 10.1.1 cost did not include any extra completedDispatches (there was one at 5.30-6.00am which is outside of the normal window). If I calculate the costs where that slot was at full rate, I get the £ 2.49 total.

Will keep an eye on it when it refreshes tomorrow to make sure any reported cost is including out of band dispatches.

[BottlecapDave]

Correct. As per the changelog for 10.1.1, there was a chance upon restart when the cost was incorrect as it was calculated before completed dispatches were retrieved.

[BottlecapDave]

I'll close this issue as the original issue seems to have been fixed. Please open a new issue if you find something different, as it helps me keeping track of issues and others who find the same problems.

[pacemaker82]

Just to be clear though, there was no restart of HA until AFTER 10.1.1 was installed. The initial incorrect price (on the old version of the integration) happened naturally.

[BottlecapDave]

That might have been OE being slow at giving completed dispatches.