Closed gcoan closed 11 months ago
Further update: At the moment batpred disabled and I re-enabled my existing flux scripts that at 17:00 discharged the batteries down to 75%. The latest batpred graph shows it proposing that best (red) is to continue the discharge in the peak period, exhaust the batteries and recharge overnight in the flux period. The base case (blue) shows the track I believe is optimal: to let the batteries continue to serve home demand until they run out early tomorrow morning when the next days solar starts filling them up again.
interestingly looking at the cost model shows that best (blue) is better than base (orange) until the early hours of the morning when the batteries are exhausted and grid import starts, and the daily home cost prediction reduces until the blue line (best) crosses the base (orange). i.e. the predicted best isn't in fact better than the base case
There are a few things to clear up here first:
However I think I might have broken something, let me have a look at Flux again.
When there's plenty of solar and the batteries are going to charge anyway I leave the inverter on a slightly lower charge rate, 1900 or so. That means I export earlier (which if I had an export limit would help to keep under it by longer trickle charging), but the primary reason is that if any house demand gets turned on it'll be met more quickly by the excess export without the inverter having to change mode. This minimises the small grid imports you naturally get throughout the day.
For force charge and discharge I do change it to 2600 and to hold the battery level I set discharge or change to zero as needed.
I may submit this as a feature enhancement at some point. Got the idea from a "The EV Puzzle" video.
I'll set calculate discharge all off and see what happens
Thanks
On 26 Sep 2023 at 19:18 +0100, Trefor Southwell @.***>, wrote:
There are a few things to clear up here first:
• Your inverter charge rate is showing not at maximum, but giving 1809w - is there any reason for that? • Calculate discharge all should be turn off (this may have some odd impacts)
However I think I might have broken something with the forward looking rates, let me have a look at Flux again. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Turning discharge all off, its still favouring doing an overnight charge, which gets me to 100% sooner:
But financially is worse off until the extra export ends up at what appears to be the same point:
I've made a fix for the issue I found, can you try this one:
https://github.com/springfall2008/batpred/releases/tag/v7.2.3
There is some odd behaviour around if the battery is going flat before 1am anyhow then it might as well export it all now at the highest rate and buy it back later which is hard to understand but it seems to depend on your expected evening load. I still need to debug that as well.
Frustratingly I think the exporting and leaving the battery flat can be the right move, why...
Export rate during peak period is marginally higher than the import rate afterwards.
Exporting from the battery involves battery discharge loss + inverter loss Powering the home involves battery discharge loss + inverter loss
Either way the battery ended up flat, so the extra export leads to a similar amount of extra import at a marginally cheaper rate.
Setting metric_min_improvement_discharge to a slightly positive value will change this calculation.
When there's plenty of solar and the batteries are going to charge anyway I leave the inverter on a slightly lower charge rate, 1900
For Predbat it's going to overwrite this anyhow, I'd suggest you leave it at maximum and let it do it's job which will control the charge amount automatically anyhow.
Thanks for such a quick turnaround Trefor, have installed 7.2.3 and it looks similar to 7.2.2 with calculate discharge all set to off
at 14:05 the best is slightly better than base, at -288.4 vs -281.5, but then after that the two predictions track the same
I understand the principle of selling the extra charge and buying it back later, probably would prefer to not unnecessarily add extra battery cycles, but think its the same issue of the import and export are too close together to be making a profit:
export (day) 19.7, import (flux cheap rate) 18.4p.
But 5% conversion losses so it really is exporting at 18.715p and importing at 19.368p
Excluding the inverter 5% loss
Our messages crossed, by the time conversion losses are included, I don't think it is cheaper to export then import on flux rates. metric min improvement discharge is already set to 0.1 (metric min improvement to 0)
You can see it's calculated around 9p better on best than base, but it's too close together to really show on the chart.
If you set metric_min_improvement to -0.1 you will see slightly less charging also.
I'm slightly confused by how your 'base' line appears to also be discharging at peak rate, can you post the battery chart again with the charge and discharge windows showing? Also do you have the blackout period (record) on the chart?
Here's the battery chart again with all the lines showing. I had turned several off to reduce the clutter, so here's the full set. If you want one with some turned off, let me know
I'm guessing that the 'base' line includes some peak rate discharging because that's what I have been doing every day by my own automation scripts prior to predbat being turned on yesterday? Just checked all the charge & discharge time slots on the inverters and they are all 00:00:00 (from my automation scripts after its finished discharging). Its not a hang-over from predbat previously running.
Here's the full home cost prediction chart, and yes, by the end of the 48 hour period the best is indeed 9p better than base, but that difference only emerges tomorrow evening around 8pm. In the period before 14:40 the base and best lines are the same.
Tomorrow evening I see the same sort of behaviour as planned for today, the battery is fully discharged in the peak period resulting in extra evening (flux daytime) import. Flux peak export 32p, but after 5% conversion loss each kW of stored battery power will only achieve 0.95kW of export, i.e. 30.4p Flux day import 30.7p, importing to meet house demand so no energy conversion, but still 0.3p/kWh more expensive than retaining the battery charge.
Is there an issue with the way the conversion losses are used in the calcs?
So as to not keep changing multiple things, I'll leave metric_min_improvement on 0 for now because it looks to me as if something isn't being calculated the way I expect it to be.
Thanks Geoffrey
Effectively it’s all in the noise, yes you get discharge losses of forced export but you also get discharge losses powering the house. Importing doesn’t model any losses.
metric min improvement says charge the battery less if it’s cheaper by X pence. A positive value means more charging and a negative value is less charging.
metric min improvement discharge says discharge the battery more if it saves you X pence. A positive value means less discharge.
I don’t account for the cost of cycling the battery but in this case it’s flat by midnight either way so it’s irrelevant.
You might find this new feature useful: https://github.com/springfall2008/batpred/releases/tag/v7.3
“ Effectively it’s all in the noise, yes you get discharge losses of forced export but you also get discharge losses powering the house. Importing doesn’t model and losses.”Ah thanks, I think this might be the nub of the issue.In what scenarios are the battery charge, discharge and inverter losses used?Agree that there’s always noise in the prediction, and in the case of other tariffs such as Go where there’s a notable difference between night and day rates, this doesn’t matter, but for Flux where the import and export rates are so close together I think it is important to model as accurately as possible.When force charging the battery and converting AC to DC there should be a loss included, and similarly on discharging DC to AC there’s another loss that occurs.In the case of flux these losses are what turns an ‘on paper’ profit actually into a small loss.I’m personally sticking to 5% but I’ve seen different figures for energy losses quoted and some people on the GivEnergy forum have reported 20-25% depending on temperature :-(Have tried changing min metric improvement to -0.1 and -0.2 and have added metric battery cycle cost at 3.5p, didn't make an appreciable difference, still doing a full peak rate discharge, emptying the battery and drawing from the grid from 7pm. I think my prediction is now being skewed by the Octopus Power up events: two hours of free electricity from 2-4pm when there’s excess power in the grid. We had one a week ago on the 20th and another yesterday.What I have been doing on these days is not letting the batteries fully charge up (stopping the 9.5 at 50% and leaving the 5.2 empty) so a lot of generated PV gets exported in the morning, then from 2pm we force charge both batteries, run the washing machine, tumble dryer, dishwasher, etc. This is appearing in the load prediction as a massive electricity consumption in the afternoon which of course if we were doing the same today would significantly drain the battery to the point that it won’t support the evening load, so predbat is exporting the remaining stored electricity during the peak period.I’ve tried changing holiday days, reducing the forecast window, increasing the cycle cost and none of them get the predictions right. I think I might need to create an extra accumulating helper that I “fill” during these events next time they happen so I can use that helper as a dummy car_charging_energy. Can’t see any obvious way of hiding such abnormal consumption behaviour ?On 26 Sep 2023, at 21:34, Trefor Southwell @.***> wrote: Effectively it’s all in the noise, yes you get discharge losses of forced export but you also get discharge losses powering the house. Importing doesn’t model and losses. metric min improvement says charge the battery less if it’s cheaper by X pence. A positive value means more charging and a negative value is less charging. metric min improvement discharge says discharge the battery more if it saves you X pence. A positive value means less discharge. I don’t account for the cost of cycling the battery but in this case it’s flat by midnight either way so it’s irrelevant.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>
When force charging the battery and converting AC to DC there should be a loss included, and similarly on discharging DC to AC there’s another loss that occurs.In the case of flux these losses are what turns an ‘on paper’ profit actually into a small loss
These losses are modelled and you can adjust them in the configuration accordingly.
What I'm saying is:
A) If you force export your battery during peak period you get 2) and then 3) after peak period B) If you power your home from the battery until it dies you get 1) and then 3) after peak period
As 1) == 2) in terms of losses and both scenarios end in an empty battery there is no difference in the losses.
As export peak is > than grid import standard then A) wins over B)
In terms of load modelling, discharging your battery doesn't change the load, as load is calculated from household consumption. If you did loads of extra stuff during that zero rate period it will impact the prediction.
The best way to smooth this out is to use the average of 3 or more different days, I use 7, 14 and 21 days ago but I keep 30 days HA history (the default is only 10 days).
Thanks for your patient explanation, this modelling of losses is exactly what I thought it would be. Previously you'd said that importing didn't model any losses so I was confused.
Following the example above, agreed, if the battery is fully exhausted at some point overnight and you are going to incur grid powering the home (3) then makes sense to export and get the better rate.
I've sorted out my problem with unusual home load behaviour skewing the prediction. I had extended my history to 30 days already and now changed days_previous to 2, 3, 4, 5, 14, 21 resulted in a much more "typical" usage prediction. I've excluded days 1, 6 and 7 as these were all power-up events where our afternoon usage was deformed.
Just to confirm, the best prediction is calculated over a now+24 hour period (forecast_plan_hours)?
Sounds good, the only thing is tomorrow 5 is what would have been 6 today.
Actually best is calculated over now + 24 hours + time until the next charge slot (up to a maximum of 48 hours). If you show 'record' on the chart you can see when the prediction ends.
Perhaps it's worth posting your new complete chart with the charge and discharge windows shown?
Follow-up to previous comment, I still think its not cost beneficial to export at flux peak only to then import at flux day rate. Exporting at 32p, but after 5% discharge loss, each kW of stored battery power generates 32*0.95=30.4p vs grid import ((no energy loss) at day rate=30.7p so not beneficial. And metric_battery_cycle_cost is set to 4p as well.
Further noting that the base load prediction doesn't now show the batteries running out overnight so there is no need to import at all in order to meet house load.
I thought I was getting towards understanding the prediction if its optimising costs over the next 24 hour period:
Next day base prediction is to reach 100% soc on PV by 14:40 which will all be at zero cost and no charging energy losses. The cost graph shows base cost at 14:40 to be -49.9p.
Next day best case prediction after a 2 hour overnight flux cheap rate topup charge reaches 100% SoC earlier at about 11:00, and at 14:40 it shows the best case cost case to be -101p.
14:40 is 24 hours on from now and so if the best case prediction is done over 24 hours then I was thinking that whilst its counter-intuitive to be doing all this when the import/export rates are so similar, but maybe I could trust it if the cost prediction correctly is taking account of energy conversion losses at the right places.
But then I took another look at the prediction path and realised that the proposal is to start the peak rate export at 16:20, and to export at full rate until the batteries are exhausted which is about 18:10. Thereafter it starts incurring peak rate import which is clearly > peak rate export and so wouldn't be cost beneficial.
Can see this on the data prediction, peak rate import starting just after 18:00
Would be better to defer starting the peak rate export, commencing it such that any grid import doesn't start until 19:00 in the day-rate period, so starting at circa 16:50?
Here's the full graphs which are showing best better than base:
Sounds good, the only thing is tomorrow 5 is what would have been 6 today.
Yes if I follow this way of excluding the days we had Octopus power up events I'll have to keep editing apps.yaml every day to shuffle the days_previous along. Not ideal. We had 3 power up events last week I think as it was quite windy which is the most we've had. 1 so far this week. There's been 4 previously since this scheme started in late August but they occur with one 1 day's notice.
If you are averaging quite a few days I doubt it really matters that much.
I don't understand why you are getting a discharge to a flat battery, if you set 'min_soc_keep' then it shouldn't be doing this.
Can you attach your appdeamon logfile for a complete cycle (5 minute)?
Ideally with Debug enabled (there's a toggle for it)
Yeah, if I average a number of days it probably won't matter much, predbat will just run slower that's all. Just got notification of another power up event tomorrow :-)
Here's the appdaemon log file with debug enabled (last run) appdaemon (1).log
I have responded to your comments on the pull request, not sure if that's routed it back to you or its still with me (I never really used github before)
Okay two things here, I suggest changing combine_discharge_slots to False to give a bit more freedom to the discharge calculation.
However setting best_soc_keep to something larger e.g. 2.0 (it's currently 0.5) will avoid a flat battery. The current 0.5 value is lower than the reserve which is the same as disabled.
One thing to note about the charts is they are only in 30 minute intervals, this is mostly due to Home Assistant complaining about the size of the data sets. In reality the battery isn't going flat until the end of the peak period.
predict best end_record 09-29 02:00:00 final soc 0.59 kwh metric -299.8 p min_soc 0.59 @ 09-27 18:55:00 kwh load 25.01 pv 36.33
Thanks. I had set combine_discharge_slots to false as that was what was recommended in the readme:
combine_discharge_slots - True # As these rates have fixed longer periods then a single slot is fine
It didn't actually make much difference.
Here with combine_discharge_slots=true and best_soc_keep=0.5:
And afterwards with combine_discharge_slots=false
But increasing best_soc_keep to 1.0 has a quite significant impact, the battery never gets flat, there's a tiny bit of peak rate discharge and then the battery gets down to 1kW and then starts charging in the cheap rate period:
And increasing best_soc_keep to 2kW:
I had progressively been dropping best_soc_keep in an attempt to stop unnecessary charging if the battery level got too low. Guessing that as I had it set to below the 4% reserve it was having other side effects.
Leaving best_soc_keep on 2kW and turning combine_discharge_slots back to True:
The best cost prediction is now showing that it doesn't start the peak export tomorrow until about 18:00, finishing it at 19:00. No peak period import is planned to occur today or tomorrow and its retaining sufficient charge to see me through to 2am both nights. image
The peak period behaviour is now much better, happy with that. I need to have another think about the overnight cheap rate charging vs let the battery charge with pv. I can see there is a benefit for some charging to occur as the soc will drop down below the best_soc_keep 'reserve' and there is further draw through the night (son playing his PC and late-night cooking), but for all the reasons said earlier not sure about there being a maximum cheap rate charge, filling up the battery with offpeak grid energy vs doing a small charge to give sufficient for the night and then letting pv charging fill the battery. Anyway, will think about that.
Here's the latest appdaemon.log if you want it appdaemon (2).log
I think the extra charging is purely to allow the discharge, if you turned off 'calculate discharge' I expect you would get less charging (but not zero as you need it for the day)?
You can also tweak metric_min_improvement to something negative e.g -0.1 to charge a little less if you like.
If I turn off calculate_discharge_first and calculate_discharge_last (calculate_discharge_all is already off), it changes the lines a little but not materially and the same activities are happening: offpeak charge from min to 3kW, this then drops down a bit, then solar charge through to about 6pm when there's a peak rate discharge, leaving enough to get through to a similar cycle the next day
I have made a new release which fixes a problem with charge calculations and thus removes calculate_charge_all option (setting it to False). It maybe worth trying this update when you get time.
If I turn off calculate_discharge_first and calculate_discharge_last (calculate_discharge_all is already off)
I really meant turning off 'calculate_discharge' so that there are no forced exports and seeing how it impacts the charge levels (as an experiment)
Have upgraded to 7.4 and the battery level seems to not be discharging; both base and best it flatlines at battery totally fulll on the prediction chart
but the cost model is changing
Here's the logfile with debug on appdaemon (4).log
I haven't changed any config items from last night,
calculate_charge_all is still showing as an entity in HA (not removed), currently set to true which is where it was before. calculate best charge, best discharge, discharge first and discharge oldest all set to true
Calculate charge oldest and calculate discharge all set to false
I really meant turning off 'calculate_discharge' so that there are no forced exports and seeing how it impacts the charge levels (as an experiment)
Confused, I don't have a calculate_discharge switch?
Sorry it was calculate best Discharge - but that's not the problem now.
Can you check your settings of 'best_soc_min' did you accidently set it to maximum last night?
Oh wait, you have your reserve set to 100%
Is 'set_reserve' turned on?
If not then you have to set it manually back to 4%
ah, the reserve probably got automatically set to 100% because I was separately doing a 2 hour force charge during the Octopus Power up event. I have not touched anything since the last message and the projection is now more 'usual' (except for discharge best which is flat lining at 14.3kW)
None of the set_ switches are turned on whilst I understand the predictions
The reserve is now 4% and mode=Eco, so was probably due to the Force Charge=2 that I'd sent to the inverter which has finished now
Looking better, if you supply the updated log I will let you know why there is no discharge tonight
Sorry it was calculate best Discharge - but that's not the problem now.
Can you check your settings of 'best_soc_min' did you accidently set it to maximum last night?
best_soc_min, margin and max are all set to 0, best_soc_keep to 2 and best_soc_step to 0.25
Probably was the reserve that was throwing it off that I wanted to leave the battery full all the time.
Now its back to more normal
turning calculate best discharge off
removes the force discharge from today and tomorrow as you'd expect. But these force discharge's are nowhere near as big as they were yesterday when it was dumping almost all of the battery out in the peak period
logfile contains:
the new 'last updated' attribute is proving useful
What do you have your battery_loss set to and your metric_battery_cycle?
Essentially less discharge I think there is less discharge as in the latest release I accounted for charging losses in the value of the charge retained in the battery itself.
I’ve set metric_battery_cycle to 3.5, battery_loss_charge and battery_loss_discharge both to 0.1 and left inverter_loss set to 0.04. I increased the battery_loss’s from 0.05 after reading some of the other people on the givenergy community forum that suggested that their loss measurements were as around 20%, 10% in, 10% out. If I change these back to 0.05 then there’s a 5.5kW peak rate force discharge, gradual load discharging to the overnight cheap period and then 8kW of charging.Curiously the base and base10 estimate is weird, its a rapid discharge this afternoon/evening and then then flat lines all the way through the night and next dayOn 28 Sep 2023, at 16:53, Trefor Southwell @.***> wrote: What do you have your battery_loss set to and your metric_battery_cycle? Essentially less discharge I think there is less discharge as in the latest release I accounted for charging losses in the value of the charge retained in the battery itself.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>
battery_loss_charge and battery_loss_discharge both to 0.1 and left inverter_loss set to 0.04.
That's giving you a total of 24% losses which I think is very high, I'd be inclined to go with 0.08, 0.08 and 0.04 making 20% total (even that maybe an over estimate)
Curiously the base and base10 estimate is weird, its a rapid discharge this afternoon/eveningCuriously the base and base10 estimate is weird, its a rapid discharge this afternoon/evening
Maybe worth looking at your predicted load chart, perhaps a spike from something in the history?
So, an update on where I'm at with predbat. I dropped the loss figures down to 0.0.8, 0.0.8 and 0.0.4 and yesterday evening I disabled my own flux peak period discharges and re-enabled predbat's set charge window, discharge window, reserve enable, reserve hold and soc enable.
I read the prediction/plan that the battery would drop down, then through the cheap period the battery level would be held so any house load would be from the grid, which seemed the 'right' behaviour; to preserve the stored battery charge until after 5am, and then be able to release it until the next day's solar started.
I got to this conclusion from seeing that the battery SoC was not increasing but home cost prediction showed that costs were rising in the 2-5am period. The logfile is I think misleading, it shows "g<bf+" which is grid importing, force charging I believe? If its holding the battery charge then could /should the status be something more distinct?
And looking at the overnight SoC this morning, that's exactly what happened, the SoC was preserved and on the Energy dashboard I can see there was 1.1kW of offpeak grid import
Then today I've been looking at the prediction for what's going to happen tonight.
Showing similar to before, a deep peak rate discharge then a big overnight cheap rate charge, and then SoC rising to 100% by mid morning. And as I said before, this leaving me wondering why this was chosen when overnight cheap is very similar to day-time export and given I'm predicted to remain above best SoC keep (set now to 1.5kW) there's no real need to import and reduce the profit potential
And then I think I've figured it out. Whilst 'best' gets to 100% SoC, 'best10' and 'base10' do not. If its significantly less sunny than predicted then the battery won't reach 100% as shown on the purple line.
Indeed yesterday and today have both been under the solcast forecast:
If I change the metric 10 weighting from 15% to 5% so much more likely favouring the solcast base forcecast than the 10% worst, the prediction plan for tonight changes significantly:
Much much less overnight cheap rate charging as there's 'more confidence' that the 100% SoC will be reached anyway, in fact just enough charging to keep the battery above best min soc after the cheap overnight period finishes.
So .... I think that I at last understand the predbat behaviours and its up to me how aggressive or not I want to be with best_soc_keep and pv_metric_10_weight
Financially the different battery behaviours doesn't actually make a lot of difference:
Sounds like it's all working as expected, which is great.
I'd recommended looking at the best10 cost, what you will see is the extra cost moves more into this scenario as the weighting reduces.
In theory best10 is the 10% scenario and hence a weighting of 0.10 (10%) is cost neutral in the long run, while high or lower than this could in theory cost you more, however a lot depends on the accuracy of the PV forecast data and your historical load data. Predbat doesn't use the SOC Keep for the 10% scenario so in effect that buffer is used up in the 10% case in most instances.
Note - some days best10 and best are very similar, other days they are a long way apart
Hopefully resolved now, feel free to re-open new issues
I will, thank you very much for all your patience and support. Currently predbat is running ok, second night 👍 On 30 Sep 2023 at 18:56 +0100, Trefor Southwell @.***>, wrote:
Hopefully resolved now, feel free to re-open new issues — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Am expecting (hoping?) that this is a non-issue and I've just set predbat up incorrectly, but I'm seeing that predbat seems to be optimising export times independently of the import times, and not planing holistically.
I'm on Octopus flux so peak rate export is much better than exporting at any other time, BUT the export rate is only slightly better than the day-time import rate, and after conversion losses (set currently to 5% but I have tried as high as 8%), it actually causes a loss if I export too much and have to import later.
[NB: there's an Octopus power-up 2 hour zero price import period manually configured 2-4pm, this is correct]
What I'm seeing is that predbat plans a big discharge all through the flux peak period, leaving the battery too empty to support the evening load and I have to import later on:![fully discharge flux peak](https://github.com/springfall2008/batpred/assets/142018870/b5fde2b7-5316-460b-afdf-66ec4c248bd1)
Flux rates are being correctly determined![export rates](https://github.com/springfall2008/batpred/assets/142018870/ee67d074-5d92-4edd-9c99-a01ef8378f3a)
I have tried to follow the recommended settings for Flux: