springfall2008 / batpred

Home battery prediction and charging automation for Home Assistant, supporting many inverter types
https://springfall2008.github.io/batpred/
95 stars 32 forks source link

Historical day warnings #1239

Open SwiftRR opened 1 week ago

SwiftRR commented 1 week ago

Describe the bug I am running the standalone predbat add on. I am experiencing continual warnings for their 'historical day' data.

I get warnings every 5 minutes as predbat updates its plan. They are nearly all the same with minor changes in minutes of number of kWh. My current predbat log spans 10 hours and has 577 of these warnings.

I always used to have such warnings using the appdaemon–predbat add on. I thought that the standalone version of predbat add-on worked better with historical day data.

Since changing to the standalone version, I have removed the appdaemon-predbat add on and I have deleted all related files.

Expected behavior No historical day warnings?

Predbat version v8.1.0 with add-on v1.1.7 Running with original HTML predbat plan card.

Environment details

Screenshots Example: from predbat add-on log

2024-06-22 15:15:07.750721: Warn: Historical day 3 has 215 minutes of gap in the data, filled from 6.64 kWh to make new average 7.8 kWh (percent 85%)
2024-06-22 15:15:07.782964: Warn: Historical day 4 has 125 minutes of gap in the data, filled from 8.4 kWh to make new average 9.2 kWh (percent 91%)
2024-06-22 15:15:07.806988: Warn: Historical day 5 has 165 minutes of gap in the data, filled from 7.5 kWh to make new average 8.47 kWh (percent 89%)
2024-06-22 15:15:07.856336: Warn: Historical day 6 has 180 minutes of gap in the data, filled from 7.52 kWh to make new average 8.6 kWh (percent 88%)
2024-06-22 15:15:07.900069: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.48 kWh to make new average 11.89 kWh (percent 97%)
2024-06-22 15:15:07.936665: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.67 kWh to make new average 10.01 kWh (percent 97%)
2024-06-22 15:15:16.522980: Info: record_status Idle
2024-06-22 15:20:07.670479: Warn: Historical day 3 has 210 minutes of gap in the data, filled from 6.66 kWh to make new average 7.8 kWh (percent 85%)
2024-06-22 15:20:07.696102: Warn: Historical day 4 has 130 minutes of gap in the data, filled from 8.4 kWh to make new average 9.23 kWh (percent 91%)
2024-06-22 15:20:07.719013: Warn: Historical day 5 has 160 minutes of gap in the data, filled from 7.5 kWh to make new average 8.44 kWh (percent 89%)
2024-06-22 15:20:07.768096: Warn: Historical day 6 has 185 minutes of gap in the data, filled from 7.51 kWh to make new average 8.62 kWh (percent 87%)
2024-06-22 15:20:07.825272: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.49 kWh to make new average 11.9 kWh (percent 97%)
2024-06-22 15:20:07.862120: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.64 kWh to make new average 9.99 kWh (percent 97%)
2024-06-22 15:22:29.002140: Info: record_status Idle
2024-06-22 15:25:09.931290: Warn: Historical day 3 has 210 minutes of gap in the data, filled from 6.68 kWh to make new average 7.82 kWh (percent 85%)
2024-06-22 15:25:09.957284: Warn: Historical day 4 has 130 minutes of gap in the data, filled from 8.4 kWh to make new average 9.24 kWh (percent 91%)
2024-06-22 15:25:09.980002: Warn: Historical day 5 has 155 minutes of gap in the data, filled from 7.5 kWh to make new average 8.4 kWh (percent 89%)
2024-06-22 15:25:10.026569: Warn: Historical day 6 has 190 minutes of gap in the data, filled from 7.5 kWh to make new average 8.64 kWh (percent 87%)
2024-06-22 15:25:10.077235: Warn: Historical day 7 has 50 minutes of gap in the data, filled from 11.5 kWh to make new average 11.91 kWh (percent 97%)
2024-06-22 15:25:10.116242: Warn: Historical day 8 has 50 minutes of gap in the data, filled from 9.61 kWh to make new average 9.96 kWh (percent 97%)
2024-06-22 15:25:19.076676: Info: record_status Idle

Log file Current predbat.log attached: predbat.log

springfall2008 commented 1 week ago

The warnings are regarding periods of zero energy usage, which its assuming is an error of recording. Maybe if you are a low user it could be possibly correct as the increments are only in 0.1 kWh I think.

gcoan commented 1 week ago

The warnings are regarding periods of zero energy usage, which its assuming is an error of recording. Maybe if you are a low user it could be possibly correct as the increments are only in 0.1 kWh I think.

Is there any way we can confirm this is definitely what's happening and what time periods it is considered to be a gap (turn debug mode on?)

Looking at my house load it is plausible, the long period on the 16th of negative house load is because my import/export meters were not enabled last weekend (givenergy support **** up) image

Need to put some logic in my template house load sensor to stop the negative/zero values if one of the underlying sensors is unavailable

2024-06-23 21:35:03.014749 INFO pred_bat: Warn: Historical day 3 has 10 minutes of gap in the data, filled from 46.1 kWh to make new average 46.43 kWh (percent 99%)
2024-06-23 21:35:03.022149 INFO pred_bat: Warn: Historical day 4 has 55 minutes of gap in the data, filled from 50.77 kWh to make new average 52.79 kWh (percent 96%)
2024-06-23 21:35:03.028686 INFO pred_bat: Warn: Historical day 5 has 5 minutes of gap in the data, filled from 21.02 kWh to make new average 21.1 kWh (percent 100%)
2024-06-23 21:35:03.038620 INFO pred_bat: Warn: Historical day 7 has 20 minutes of gap in the data, filled from 46.56 kWh to make new average 47.22 kWh (percent 99%)
2024-06-23 21:35:03.045531 INFO pred_bat: Warn: Historical day 8 has 210 minutes of gap in the data, filled from 68.81 kWh to make new average 80.56 kWh (percent 85%)

I can add something to the documentation if negative/zero load does cause this message

SwiftRR commented 1 week ago

I have switched on 'debug enable' for 30 minutes from 10:30 pm to 11:00 pm on 23rd June. The debug enable seems to have stripped out many of the historical day warnings from predbat.log. If this shows up for periods of zero energy usage, I have been getting 6 warnings every 5 minutes like clockwork. That is 1728 per day!

This is a clean install of the standalone predbat and I have deleted anything associated with appdaemon.

I have attached my predbat.log, predbat add on log and one of the debug files from this evening. predbat add on log.txt predbat.log predbat_debug_22_55_00.yaml.txt

Predbat is running smoothly but the logs are getting swamped by the warnings, especially the predbat add on log. If there is nothing untoward with these warnings, should they be suppressed?

This is the relevant section from my apps.yaml file:

days_previous:
    - 2
    - 3
    - 4
    - 5
    - 6
    - 7
    - 8

  days_previous_weight:
    - 1

Rob

cyberkryten commented 1 week ago

I'm also seeing warnings which I assume are the cause of Predbat thinking usage is pretty much zero for many hours overnight. The absolute minimum draw even when the house is empty is 800-900W overnight so they are not correct that there is zero draw. They also often decrease in duration and don't seem to be consistent from one day to the next:

` 2024-06-24 13:35:02.806266 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.39 kWh to make new average 66.47 kWh (percent 88%) 2024-06-24 13:35:02.810929 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 58.29 kWh to make new average 59.75 kWh (percent 98%) 2024-06-24 13:40:02.816092 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.54 kWh to make new average 66.64 kWh (percent 88%) 2024-06-24 13:40:02.820667 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 58.1 kWh to make new average 59.55 kWh (percent 98%) 2024-06-24 13:45:02.770606 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.75 kWh to make new average 66.88 kWh (percent 88%) 2024-06-24 13:45:02.774912 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.76 kWh to make new average 59.2 kWh (percent 98%) 2024-06-24 13:50:02.830270 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.71 kWh to make new average 66.83 kWh (percent 88%) 2024-06-24 13:50:02.835167 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.45 kWh to make new average 58.88 kWh (percent 98%) 2024-06-24 13:55:02.773380 INFO pred_bat: WARN: Historical day 3 has 175 minutes of gap in the data, filled from 58.94 kWh to make new average 67.1 kWh (percent 88%) 2024-06-24 13:55:02.777915 INFO pred_bat: WARN: Historical day 7 has 35 minutes of gap in the data, filled from 57.13 kWh to make new average 58.55 kWh (percent 98%)

2024-06-25 13:35:02.231452 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 51.64 kWh to make new average 53.5 kWh (percent 97%) 2024-06-25 13:35:02.234065 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.39 kWh to make new average 66.47 kWh (percent 88%) 2024-06-25 13:40:02.212376 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 51.18 kWh to make new average 53.02 kWh (percent 97%) 2024-06-25 13:40:02.215064 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.54 kWh to make new average 66.64 kWh (percent 88%) 2024-06-25 13:45:02.735502 INFO pred_bat: WARN: Historical day 3 has 50 minutes of gap in the data, filled from 50.69 kWh to make new average 52.51 kWh (percent 97%) 2024-06-25 13:45:02.738258 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.75 kWh to make new average 66.88 kWh (percent 88%) 2024-06-25 13:50:02.741152 INFO pred_bat: WARN: Historical day 3 has 55 minutes of gap in the data, filled from 50.43 kWh to make new average 52.43 kWh (percent 96%) 2024-06-25 13:50:02.743978 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.71 kWh to make new average 66.83 kWh (percent 88%) 2024-06-25 13:55:02.718396 INFO pred_bat: WARN: Historical day 3 has 60 minutes of gap in the data, filled from 49.92 kWh to make new average 52.09 kWh (percent 96%) 2024-06-25 13:55:02.721249 INFO pred_bat: WARN: Historical day 4 has 175 minutes of gap in the data, filled from 58.94 kWh to make new average 67.1 kWh (percent 88%) `

Is there an additional debug level that displays the actual times that are missing and which sensor it thinks is blank?

mpartington commented 1 week ago

Have you checked the load entity from GivTCP for the dates affected to rule that in/out?

I suspect you may be seeing something like mine below, where the load energy value doesn't seem to increment correctly. This is either a GivTCP or inverter firmware bug in my view, I would get lots of missing data warnings as zero usage in an hour isn't realistic.

To fix, I ended up creating 2 helpers. The first integrates the GivTCP power entity, I then feed this into a utility meter. This seems to work/increment better than the GivTCP provided entity. I then set this in apps.yaml.

Alternatively there is a bug

Screenshot_2024-06-25-14-14-47-603_io homeassistant companion android