ERROR: Unable to fetch history for sensor.calc_house_load_today

[gcoan]

Describe the bug Installed the predbat+appdaemon combined add-on to play with the install process. Had stopped appdaemon beforehand. On restarting appdaemon, predbat crashed with an errors:

2024-01-01 23:30:00.766018 INFO pred_bat: Predbat mode is set to Control charge & discharge 2024-01-01 23:30:10.853540 WARNING pred_bat: Coroutine (<coroutine object Hass.get_history at 0x7f06d0477e40>) took too long (10 seconds), cancelling the task... 2024-01-01 23:30:10.857103 INFO pred_bat: ERROR: Unable to fetch history for sensor.calc_house_load_today 2024-01-01 23:30:14.642728 INFO pred_bat: ERROR: Exception raised

And on the 3rd run, worked perfectly??

2024-01-01 23:40:00.061192 INFO pred_bat: Predbat mode is set to Control charge & discharge 2024-01-01 23:40:01.558133 INFO pred_bat: Found 12961 load_today datapoints going back 9 days 2024-01-01 23:40:05.152818 INFO pred_bat: Car charging hold False threshold 6.0

Expected behavior Wouldn't expect predbat to crash on loading history data, even if there's a lot of it/

Predbat version 7.14.29

Environment details HAOS Custom load_today template sensor (as I can't use the givtcp inverter load's with two hybrid inverters, they are both wrong as the load is shared between them)

Log file appdaemon (8).log

[gcoan]

Further update: predbat ran through OK the 3rd time, setting the inverter to charging OK, but when it next ran 5 minutes later, it crashed again, same error

[gcoan]

I can't work out why this is happening.

Looking back in the logfiles, predbat was loading the same 12961 datapoints over 9 days no problem at all yesterday appdaemon.log.9.txt

But toiday it works sometimes but more often doesn't.

Restarted HA & the add-on's, same pattern. Uninstalled appdaemon+predbat add-on, same pattern. Shutdown the entire HASS VM and restarted it, same thing, sometimes works, sometimes doesn't.

Deleted one day of history out of days_previous (day 8), thought I'd cracked it as it loaded 8 days history and ran fine for the next two 5 minute cycles, then crashed the next 2 times.

Deleted day 2 out, crashed again. Put day 2 back and it worked fine for that run but failed the next time around.

Stripped it back to just load days 2 and 3 and it worked for the next 2 cycles so I'll leave it like that for now, but still weird appdaemon (9).log

[springfall2008]

Hmmm, I think it's probably a co-incidence. I've had other users with the same issues and it is usually down to an overloaded machine.

I take it you haven't accidentally had two instances of Predbat running (one in appdaemon-predbat and one in appdeamon)?

How much history do you keep for HA and what type of hardware are you using?

I don't know if the 10 second limit can be changed, I'll have a google.

[gcoan]

I did briefly have two instances running but stopped the app daemon-predbat when I realised it was running. I have now uninstalled appdaemon-predbat in case that was a trigger but I’m not sure it was. It’s very strange, ran all night ok with 2 days_previous, I added two more, it ran fine a couple of times and then crashed the same way. Dropping it back to 3 days_previous it seems ok for now. HA is set to 14 days history and I haven’t changed that for ages. Also haven’t applied any upgrades in the last few days. I’m running as a HAOS VM on an Intel i5 windows pc. The VM is set to 2 cpu and memory to 3gb I think - both the max I could set.

On 2 Jan 2024, at 08:18, Trefor Southwell @.***> wrote:

 Hmmm, I think it's probably a co-incidence. I've had other users with the same issues and it is usually down to an overloaded machine.

I take it you haven't accidentally had two instances of Predbat running (one in appdaemon-predbat and one in appdeamon)?

How much history do you keep for HA and what type of hardware are you using?

I don't know if the 10 second limit can be changed, I'll have a google.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[gcoan]

I had completely restarted HA and the add-on's more than once, and the VM last night to see if that cured it.

Just now I have shut the VM down and rebooted the surrounding windows PC. Increased the days_previous to days 2-7 and its running OK at the moment. The HA system/hardware view shows one CPU spike to 60% but running at about 30% all the time. Memory peaking at 65% 2Gb of 3Gb. So busy but not excessive or swapping out

[springfall2008]

Kind of odd, I've only had it during backups. I think there is a fix I can make using async and await but it's a little messy.

[gcoan]

Its very odd. After restarting the whole PC that HASS is VM'd on, it ran absolutely stable all afternoon with days_previous set to -2 ... -6

I changed it to add -7 and -8 back in, it ran fine a couple of times and then repeatedly kept crashing loading the load_today data. Looking at the CPU and memory around the time, it was peaking CPU at around 68% and memory at 89% at the 5 minute points when predbat was failing, but I had some CPU and memory spikes a little higher in between times, but never to maximum. Looking at the add-on's, I found ESPHome was running that I shutdown, but otherwise it was all still things I have had running for ages.

Deleting -7 and -8 out of days_previous and it ran the first time OK then kept crashing.

appdaemon.log.1 (3).txt

No real idea what the problem is.

Maybe there is a corruption in the template sensor history that sometimes is causing the load to time out, but I can't see how to find and fix this. And why it sometimes works on certain days history and other times doesn't, no idea.

I need the template sensor as I have two inverters and load is shared between the two inverters, and I have a FIT array, so to get the most accurate house load I have to calculate it myself. But at the moment I am only actually using 1 battery on 1 of the inverters so I changed the apps.yaml to pickup the givctp load data from that inverter as comparing that to my calculated load sensor, its close enough, under-cooked a bit each day, but should only be out by the PV from the other inverter and the FIT array.

Put that back in apps.yaml and its working perfectly - i.,e. not crashing. It is reporting a couple of gaps in the history that its having to fill - my template sensor didn't have these issues, but these are only warnings 2024-01-02 21:44:10.709369 INFO pred_bat: WARN: Historical day 5 has 5 minutes of gap in the data, filled from 51.29 kWh to make new average 51.47 kWh (percent 100%) 2024-01-02 21:44:10.714793 INFO pred_bat: WARN: Historical day 6 has 85 minutes of gap in the data, filled from 46.3 kWh to make new average 49.21 kWh (percent 94%) 2024-01-02 21:44:10.724790 INFO pred_bat: WARN: Historical day 8 has 15 minutes of gap in the data, filled from 58.17 kWh to make new average 58.78 kWh (percent 99%)

But I have been running with my template sensor and days 2-8 days_previous for a month or so with no issues. Don't understand

[springfall2008]

I may have a fix for this now with a new task to fetch history and an error thrown if it fails, please file a new ticket if need be

[gcoan]

After I had a spate of failures loading the custom sensor I swapped back to the inverter house load sensor, which worked fine, and now I’m back on the custom sensor again.So whether it was something in the database that was causing it to randomly be slow loading, I don’t know.Anyway OK now plus the async loading option as wellThanks On 3 Feb 2024, at 13:07, Trefor Southwell @.***> wrote: I may have a fix for this now with a new task to fetch history and an error thrown if it fails, please file a new ticket if need be

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>