HOLD CHARGE ---- Predbat stopping battery from discharging despite high unit rate

[mpartington]

Describe the bug Currently in the peak period of Octopus agile and the unit rate is high. Predbat has chosen to HoldChrg for the first hour of the period (16.00 to 17:00). This is probably because my predicted load is well below my PV generation, so it believes it is adequately covered. However this approach is ### unnecessarily risky:

1. If load is unexpectedly high and exceeds the solar, I will be charged at a high rate - in fact this would have happened if I hadn't forced to ECO, as we were cooking earlier than usual with high usage
2. If the PV generation is lower than predicted, I will be charged at a high rate

Additionally the battery is sat at 100% and will easily reach times where more effective rates are available to charge, so no need to hold.

EDIT... this will affect logic for both Hold charge and Charge Freeze. There also appears to be a case where the battery is at 100%, but the plan and status just says 'charge', so I think that would also need review.

Expected behavior Predbat to allow battery discharge at high rates, in case of unexpectedly high usage, loads or poorer solar generation There is really no risk in this approach, either:

1. The solar exceeds load, in which case the battery continues to charge (or in my case export excess solar), as per todays logic, or
2. Load exceeds solar generation (for whatever reason) and the battery covers the extra load. This is more cost effective than paying 28p/kWh

Predbat version

v7.16.10

Screenshots

Log file

Relevant part of the log will be 16:00 onwards (30-March). I used the override to force the battery to eco mode once I spotted the behaviour appdaemon.log

Second log file to include the third occurrence during 6pm slot appdaemon.log

[mpartington]

Thinking about this further, the.only scenario this is appropriate for is if it is if either the grid unit price is lower than the export price and it is cheaper to fill the battery at 15p (cost of lost export) than a future slot price.

Another option is if you would either want to stop the battery charging to force export. This is done by freeze discharge (but freeze discharge allows discharge).

All other times when grid unit price is high, best to leave in ECO mode, that way it will continue to charge, or export, or provide battery power to the house.

Cloud modelling partly mitigates lower than expected PV, but there is no way for Predbat to know I want to suddenly pull 6kW as am cooking lunch earlier. As said, if unit price was say 17p, the current approach is valid, but not at 28p

[mpartington]

And again, at even higher rate and no solar

[gcoan]

What's your best soc keep and best soc min set to ?

Also what are the inverter losses?

Very strange that it's basically preserving the soc all evening.

I assume that the holiday mode in the status was an error?

[mpartington]

Soc min 0 Soc keep 0.5 Battery metric 1p Inverter loss 4% Charge loss 3% Discharge loss 6% Holiday mode was a quick trial

However I don't think it's related to any of those. Now there is no solar, it is happy to discharge battery on 21p slot.

I'm sure it's because the average solar production during the slot is greater or equal to the predicted load. So Predbat assumes no need for the battery. However, for high rates, the battery should just be set to eco, as it gives the same result with added benefit that it uses the battery if needed

[gcoan]

Yes none of those are causing the battery to be held.

As you say, it's almost certainly that your PV prediction is above your super low consumption. Mine is 1kW or more every slot! Admittedly I've got an ASHP but even overnight mine is 0.3-0.5 a slot.

Selecting Eco would be a better solution from predbat. The 1p battery metric is having an impact in borderline discharge planning but not in the peak.

One fix you could do is put an import rate adjustment for the peak period with a high load adjustment factor which would force predbat to put the battery in eco rather than relying on pv

[mpartington]

Yeah, I did have the battery metric set to 0.5, but my settings got lost on a HA restart (happens occasionally) and I loaded an old saved profile. Now set it back to 0.5.

I wondered about adding an offset, on import rate adjustment, but there should already be plenty of difference in cost. Adding the load factor is a good idea, until hopefully Trefor can tweak the logic

[RobinCu]

I have an automation that calls a service to toggle Set Charge Freeze on/off. My triggers are simpler as I'm on IOG, so mine avoids drawing from the grid at peak rate of 30.6p regardless of PV/load. You might be able to do something similar around import / export metrics.

[mpartington]

I have an automation that calls a service to toggle Set Charge Freeze on/off. My triggers are simpler as I'm on IOG, so mine avoids drawing from the grid at peak rate of 30.6p regardless of PV/load. You might be able to do something similar around import / export metrics.

Another great idea, I don't like freeze charge, so could turn it off if it triggers on above say 17 p slot

[mpartington]

I can't see what is being changed in GivTCP to put it into hold charge mode (and therefore what to reverse). Is nothing changing because it's being commanded via REST?

Only issue is I imagine I would be fighting it every 5 mins, so maybe Geoffrey's solution might be best

[RobinCu]

I'm guessing it sets the discharge rate to 0W? I'm on my phone so can't check logs easily.

If toggling on/off based on price, it should only run every 30 minutes max.

[gcoan]

I'm guessing it sets the discharge rate to 0W? I'm on my phone so can't check logs easily.

If toggling on/off based on price, it should only run every 30 minutes max.

Predbat will run every 5 minutes and if the inverter isn't as it expects it to be, it'll change it back to the plan

[mpartington]

Looks like it's leaving charge enabled, then changes the charge slot end time to the end of the current slot. I should (I think) just be able to turn enable charge schedule off as an interim fix.

Although if it does get reset every 5, I'm back to the load scaling method, which may be harder to generically implement

[RobinCu]

Predbat will run every 5 minutes and if the inverter isn't as it expects it to be, it'll change it back to the plan

Yes, perhaps I'm not explaining very well tapping out on the GitHub app or misunderstanding the issue!
I'm not sure why the inverter would be out of sync with Predbat, as I'm only changing Predbat settings.
If, for example, the import rate changes from 12p to 25p, the automation (set at 17p) would switch Set Charge Freeze is off, Predbat would recalc at the start of the 30-minute slot, and would likely switch to Idle (Eco). It could actually select any status really, except Hold Charge / Freeze Charge.

[gcoan]

Predbat will run every 5 minutes and if the inverter isn't as it expects it to be, it'll change it back to the plan

Yes, perhaps I'm not explaining very well tapping out on the GitHub app or misunderstanding the issue!
I'm not sure why the inverter would be out of sync with Predbat, as I'm only changing Predbat settings.
If, for example, the import rate changes from 12p to 25p, the automation (set at 17p) would switch Set Charge Freeze is off, Predbat would recalc at the start of the 30-minute slot, and would likely switch to Idle (Eco). It could actually select any status really, except Hold Charge / Freeze Charge.

Yes you are absolutely right, if all you are doing is changing the predbat set charge freeze switch off and on then predbat will change the plan accordingly and then change the inverter. I thought the suggestion was to directly change the discharge rate back from zero to 2600 or whatever which predbat would then override on its next 5 minute cycle.

My suggestion of the load override in the peak period was specifically to address the peak import issue. If the concern is that predbat is setting freeze discharge when it shouldn't as it any unexpected high load will cause grid import, then maybe just leave the switch off all the time?

[mpartington]

Completely forgot there was a charge freeze option! Simple fix then, i'll turn it off :)

On Sat, 30 Mar 2024, 20:51 Geoffrey Coan, @.***> wrote:

Predbat will run every 5 minutes and if the inverter isn't as it expects it to be, it'll change it back to the plan

Yes, perhaps I'm not explaining very well tapping out on the GitHub app or misunderstanding the issue! I'm not sure why the inverter would be out of sync with Predbat, as I'm only changing Predbat settings. If, for example, the import rate changes from 12p to 25p, the automation (set at 17p) would switch Set Charge Freeze is off, Predbat would recalc at the start of the 30-minute slot, and would likely switch to Idle (Eco). It could actually select any status really, except Hold Charge / Freeze Charge.

Yes you are absolutely right, if all you are doing is changing the predbat set charge freeze switch off and on then predbat will change the plan accordingly and then change the inverter. I thought the suggestion was to directly change the discharge rate back from zero to 2600 or whatever which predbat would then override on its next 5 minute cycle.

My suggestion of the load override in the peak period was specifically to address the peak import issue. If the concern is that predbat is setting freeze discharge when it shouldn't as it any unexpected high load will cause grid import, then maybe just leave the switch off all the time?

— Reply to this email directly, view it on GitHub https://github.com/springfall2008/batpred/issues/897#issuecomment-2028466144, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZQHXHLSQLP332M46PMHH6DY24JUHAVCNFSM6AAAAABFPV2AMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGQ3DMMJUGQ . You are receiving this because you authored the thread.Message ID: @.***>

[mpartington]

Same issue on the GivTCP Facebook group

[mpartington]

Unfortunately it appears turning off 'Set Charge Freeze' does not impact HoldCharge, which must be hard coded. Looks like it will need Trefor to tweak the logic slightly. Now trying an automation for load scaling in peak window. However I imagine that has dangers, as it may hypothetically try to fill the battery at say 25p early in peak slot to avoid 40p later.

[mpartington]

Cracked it, can set the slots to idle automatically. Not ideal, but a reasonable containment

[gcoan]

I've looked through the customisation controls and there doesn't seem to be any way to prevent this unless you make some major changes to your PV or PV10% scaling which will impact the rest of your plan.

Manual force overrides will work, just need to trigger the automation when you're at least 18 hours away from the time period so that the required slots appear in the selector.

Alternative using a rate override in apps.yaml:

rates_import_override:

• start: '15:00:00' end: '20:00:00' load_scaling: 3.0

Means the predicted load will be three times higher in the extended peak period. I'd put it a bit longer just in case you start cooking early

[mpartington]

Thanks, setting the slots.to.ECO should fix it within the peak period, but if that fails then I'll give rates scaling a go

https://pastebin.com/ipYPr9dQ

[mpartington]

More Hold Charge oddity

In my eyes, the only reason for hold charge is because it is cheaper to use the grid than discharge the battery.

So why am I hold charging at higher rates and discharging the battery when the rates are lower? 12.13p for example is lower than my 15p export rate. So it's cheaper to use grid than effectively a 15p unit rate to recharge from Solar (+AC3 losses) to top the battery back up.

[gcoan]

I'm not getting hold charge at all in my plan today, but the battery is held at 100% and is in Idle mode (although some freeze discharge later)

Makes me wonder if some of the plan evaluation switches (second pass etc) might be different between our plans?

Answering your question about why the battery is not discharging: the battery is being held throughout at 100% because your PV is higher than your house load so the excess is being exported. Why it starts discharging in the last slot, I don't know, would need to see the evening predictions. I have found Predbat has a tendency to discharge just before the peak whereas I'd rather keep it just in case so I put an export rate adjustment from 15:00 to 20:00 to stop this.

[mpartington]

The only difference I can see is I have discharge on charge slots switched on. The plan has updated and currently makes sense for rest of day. The only time I would disagree with holding charge under 17.25p (15p export x 15% i.e. the round trip losses, per my settings), is if Predbat knew it didn't need to maintain current soc as there was a very cheap charging window around the corner.

I think Trefor has mentioned on FB that it needs a fix, so happy to wait it out until it reaches the top of the wait list. Hopefully he's enjoying the Easter break.

What I might do is if maybe automate if slot price >x, and charging is switched on, then force set to idle. Just got to work out how to return the current time in xx:00:00 format. Let's face it, by the time I've done that, Trefor will probably have fixed it 😆

[gcoan]

My home consumption is much higher than yours so I don't have the Hold charging occurring when PV>Load, but I do have an issue about the discharge that's logged under #861.

My issue is that Predbat starts discharging when its not cost effective to do so, discharging the battery at the same fixed 15p, resulting in the soc being drained and future slots then having to grid import. After losses my effective export rate is 13.68p, so to my mind it shouldn't be discharging and leaving the SoC requiring import at anything higher than this.

I'm finding I am often having to put force Idle's in to preserve the battery level to get through the peak and beyond. Not tried to automate it yet, hoping Trefor will be able to change the behaviour. Manipulating date/time in Jinja is painful. I do a bit of it in my dashboards to work out when to put the washing machine & dishwasher on based on cheap rate slots. Prefer not to have to do more in an automation !

[mpartington]

Yes, I think I've been seeing that too. Seems to do a lot of discharge, then filling up on marginal slots later. I've been gradually increasing the discharge metric and am currently at 3p. I might raise it further

On Mon, 1 Apr 2024, 16:15 Geoffrey Coan, @.***> wrote:

My home consumption is much higher than yours so I don't have the Hold charging occurring when PV>Load, but I do have an issue that's partially logged under #861 https://github.com/springfall2008/batpred/issues/861.

My issue is that Predbat starts discharging when its not cost effective to do so, discharging the battery at the same fixed 15p, resulting in the soc being drained and future slots then having to grid import. After losses my effective export rate is 13.68p, so to my mind it shouldn't be discharging and leaving the SoC requiring import at anything higher than this.

I'm finding I am often having to put force Idle's in to preserve the battery level to get through the peak and beyond. Not tried to automate it yet, hoping Trefor will be able to change the behaviour. Manipulating date/time in Jinja is painful. I do a bit of it in my dashboards to work out when to put the washing machine & dishwasher on based on cheap rate slots. Prefer not to have to do more in an automation !

— Reply to this email directly, view it on GitHub https://github.com/springfall2008/batpred/issues/897#issuecomment-2029928969, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZQHXHPCRRSSW7OFFC3GFYDY3F22HAVCNFSM6AAAAABFPV2AMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZHEZDQOJWHE . You are receiving this because you authored the thread.Message ID: @.***>

[mpartington]

Just realised this is a very similar/ same issue as #858 . Perhaps covers more soc than just 100%

[mpartington]

@springfall2008 as requested, the log from today is attached in this comment. There are also some logs in the OP

appdaemon.log

[mpartington]

appdaemon.log

Log from this morning, where there is inconsistent implementation between similar slots, suggesting a bug. In all cases the solar is greater than predicted load (which seems required for current implementation), but not all slots are HOLD CHARGE. There is also no pattern, with neither the cheapest or most expensive slots chosen.

In this scenario, I would agree that HOLD CHARGE is appropriate for all the slots shown in green, as the cost to refill the battery at my (loss of) 15p export rate is more expensive than running directly from grid.

[mpartington]

appdaemon.log

[mpartington]

appdaemon.log With debug (version on main)

[springfall2008]

So somehow it actually thinks it's cheaper which is odd...

2024-04-18 09:21:37.816165 INFO pred_bat: Try optimising charge window(s) 6: 04-18 12:00:00 - 04-18 12:30:00 price 14.6 cost 152.54 metric 127.69 keep 0 selected 8.36 was 8.36 results {8.36: 127.69, 7.86: 127.71, 0.0: 127.7, 0.33: 127.81}

[mpartington]

Is it because in this instance setting it to 100% or setting the slot at idle will give the same theoretical answer as the solar covers.load.

Ironically (and opposite to the peak rates issue) using the battery for any reason would actually cost more for the majority of those green slots, so a hold.charge would be appropriate.

[springfall2008]

If the 100% is the same cost as 0% then it will select 0%.

The problem here is that the 100% reported as cheaper.

I've pushed another potential fix to main, no idea if it will help, maybe give it a try?

[mpartington]

Ok, thanks. Is this hopefully fixing both.the hold at high rates (original issue)and the seemingly random hold at lower.rates (pictures posted today) or just part of it?

An unfortunate byproduct of more low rates tomorrow is there won't be as many opportunities for it to schedule a hold charge

[springfall2008]

Ok, thanks. Is this hopefully fixing both.the hold at high rates (original issue)and the seemingly random hold at lower.rates (pictures posted today) or just part of it?

An unfortunate byproduct of more low rates tomorrow is there won't be as many opportunities for it to schedule a hold charge

I don't know if it will fix anything, but it might as it was making charging to 100% more attractive then ECO mode.

[mpartington]

Ok, thanks. Is this hopefully fixing both.the hold at high rates (original issue)and the seemingly random hold at lower.rates (pictures posted today) or just part of it? An unfortunate byproduct of more low rates tomorrow is there won't be as many opportunities for it to schedule a hold charge

I don't know if it will fix anything, but it might as it was making charging to 100% more attractive then ECO mode.

Probably worth releasing, I haven't seen any dodgy hold charges on this version....yet . I'll keep monitoring though. By the way, have you fixed the car charging slots issue as well? Mine has been absolutely fine since I updated to main

[springfall2008]

Ok, thanks. Is this hopefully fixing both.the hold at high rates (original issue)and the seemingly random hold at lower.rates (pictures posted today) or just part of it? An unfortunate byproduct of more low rates tomorrow is there won't be as many opportunities for it to schedule a hold charge

I don't know if it will fix anything, but it might as it was making charging to 100% more attractive then ECO mode.

Probably worth releasing, I haven't seen any dodgy hold charges on this version....yet . I'll keep monitoring though. By the way, have you fixed the car charging slots issue as well? Mine has been absolutely fine since I updated to main

I've not fixed the car charging slots I'm afraid, that's one for another day!

[mpartington]

Strange, the car slots seem to have started working.

On Fri, 19 Apr 2024, 19:53 Trefor Southwell, @.***> wrote:

Ok, thanks. Is this hopefully fixing both.the hold at high rates (original issue)and the seemingly random hold at lower.rates (pictures posted today) or just part of it? An unfortunate byproduct of more low rates tomorrow is there won't be as many opportunities for it to schedule a hold charge

I don't know if it will fix anything, but it might as it was making charging to 100% more attractive then ECO mode.

Probably worth releasing, I haven't seen any dodgy hold charges on this version....yet . I'll keep monitoring though. By the way, have you fixed the car charging slots issue as well? Mine has been absolutely fine since I updated to main

I've not fixed the car charging slots I'm afraid, that's one for another day!

— Reply to this email directly, view it on GitHub https://github.com/springfall2008/batpred/issues/897#issuecomment-2067120710, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZQHXHIG3LP4JD3ZBRKHD6LY6FR37AVCNFSM6AAAAABFPV2AMSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXGEZDANZRGA . You are receiving this because you authored the thread.Message ID: @.***>

[mpartington]

appdaemon.log

appdaemon_2.log

CHARGEHOLD set during 16:00 slot at peak rate. Hopefully the log contains enough into (only turned debug on shortly before the next run was due)

[mpartington]

And the 16.30 slot has also set, so here's another debug log with at least 3 runs

appdaemon.log

[springfall2008]

Interesting the cost is the same at 100% vs 0% but the metric is "cheaper" on the 100% - need to figure out why:

2024-04-20 16:47:20.712666 INFO pred_bat: Optimise price set 28.9 price 28.9 start_at_low True best_price 14.9 best_metric 353.41 best_cost 403.8
2024-04-20 16:47:20.853378 INFO pred_bat: Sim: SOC 8.36 window 0 metricmid 403.8042983569447 metric10 668.8117729229863 soc 8.359748293505861 soc10 8.353849287983877 final_iboost 0.0 final_iboost10 0.0 metric_keep 0 metric_keep10
2024-04-20 16:47:20.857924 INFO pred_bat: Sim: SOC 8.36 window 0 imp bat 52.4 house 7.82 exp 7.22 min_soc 0.33 @ 04-21 20:45:00 soc 8.36 cost 403.8 metric 352.91 keep 0 metricmid 403.8 metric10 608.12
2024-04-20 16:47:20.862788 INFO pred_bat: Sim: SOC 7.86 window 0 metricmid 403.8042983569447 metric10 668.8117729229863 soc 8.359748293505861 soc10 8.353849287983877 final_iboost 0.0 final_iboost10 0.0 metric_keep 0 metric_keep10
2024-04-20 16:47:20.868435 INFO pred_bat: Sim: SOC 7.86 window 0 imp bat 52.4 house 7.82 exp 7.22 min_soc 0.33 @ 04-21 20:45:00 soc 8.36 cost 403.8 metric 353.42 keep 0 metricmid 403.8 metric10 608.12
2024-04-20 16:47:20.872682 INFO pred_bat: Sim: SOC 0.0 window 0 metricmid 403.8042983569447 metric10 668.8117729229863 soc 8.359748293505861 soc10 8.353849287983877 final_iboost 0.0 final_iboost10 0.0 metric_keep 0 metric_keep10
2024-04-20 16:47:20.878630 INFO pred_bat: Sim: SOC 0.0 window 0 imp bat 52.4 house 7.82 exp 7.22 min_soc 0.33 @ 04-21 20:45:00 soc 8.36 cost 403.8 metric 353.41 keep 0 metricmid 403.8 metric10 608.12

[mpartington]

Glad you understand what you are looking at 😆

I wonder if there are 2 issues

1. Hold charge is sometimes scheduled (or not scheduled) incorrectly
2. At Peak rates, theoretically the cost is balanced based on prediction, but it does not : i) Adequately cover the risk of higher real world loads, meaning a greater proportion of peak rate would.be used ii) Assumes solar and load is relatively linear across the period, whereas it could oscillate above/below the mean assumption. Again peak prices would be needed to cover.

Fixing 1 may not fix 2, because it is based on prediction. It needs a second leg of logic (assuming there is sufficient battery capacity to get to a lower rate), that hold charge is never applied above a certain unit threshold

[springfall2008]

Okay I have some code to try on 'main' which includes a rounding fix and some more debug to capture what is going on if it doesn't work (change is https://github.com/springfall2008/batpred/pull/983)

1. I think this maybe the rounding issue but we will see
2. i) Yes it's kind of right, I was wondering about a load modulation model to help with this, although PV10 scenario kind of does this its too linear
3. ii) This is cloud model so it shouldn't be

[mpartington]

Great, I'll give it a go. I do think it's important to address 2.i. somehow, as unless you have the same load profile as is predicted, this will always be a problem. That's why I thought a unit cost base may work.

[mpartington]

Debug for new version on main appdaemon.log Edited to add a file with few more runs appdaemon.log

[springfall2008]

Tracked down the issue, a weighting of 0.5 was added to match the curent soc target against window 0, but this applied event if not charging. Changed to only apply if charging currently.

Added to branch: https://github.com/springfall2008/batpred/tree/loadscale

[springfall2008]

https://github.com/springfall2008/batpred/pull/984

[mpartington]

Logs from this morning appdaemon.log appdaemon1.log

[mpartington]

More data (no debug ). Possible freeze charge at 16.32 appdaemon.log

[springfall2008]

Not sure about that

Charge window will be: 2024-04-22 16:30:00+01:00 - 2024-04-22 17:00:00+01:00 - current soc 85 target 82 Inverter 0 Current charge limit is 100 % and new target is 82 %

So its actually like a 'no charge' unless the SOC would fall below 82%?