search

mampfes / ha_epex_spot

Adds EPEX Spot data to Home Assistant.
MIT License
130 stars 19 forks source link

StopIteration Exception breaks entity updates until HASS restart #16

Closed fblaese closed 1 year ago

fblaese commented 1 year ago

I got the following exception at midnight, which breaks the update timer and does not do any future updates until HASS is restarted:

Error doing job: Exception in callback async_track_point_in_utc_time.<locals>.run_action(<Job HassJobT...7f84cb97dbd0>>) at /usr/src/homeassistant/homeassistant/helpers/event.py:1310
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1325, in run_action
    hass.async_run_hass_job(job, utc_point_in_time)
  File "/usr/src/homeassistant/homeassistant/core.py", line 590, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1553, in pattern_time_change_listener
    hass.async_run_hass_job(job, dt_util.as_local(now) if local else now)
  File "/usr/src/homeassistant/homeassistant/core.py", line 590, in async_run_hass_job
    hassjob.target(*args)
  File "/config/custom_components/epex_spot/__init__.py", line 189, in _on_hour_change
    source.update_time()
  File "/config/custom_components/epex_spot/__init__.py", line 94, in update_time
    self._marketdata_now = next(
StopIteration

It does not happen every night, but it did happen at least three times in the last two weeks.

mampfes commented 1 year ago

Hm, that is interesting. Does this happen always at the same time? (because update_time is called every hour to update the current values). Which data source (Awattar or EPEX Spot) and which market area are you using?

What the code is trying to do is to get the values for the current hour. There is data available for other time slots - otherwise it would return earlier in line 89. But it doesn't find data for the current hour. So, we have to find out why the data is missing.

fblaese commented 1 year ago

I am using both data sources. As far as I can tell, it always happened at midnight (00:00). I took a quick peek at the code, but I couldn't find the cause of this issue right away. If I find some time, I will try to dig a little deeper.

Would it be a good idea to catch this exception in case there isn't any data? Because this exception would probably also be raised if the integration was unable to fetch new data (e.g. no internet), wouldn't it?

fblaese commented 1 year ago

Actually, I do think that the issue is caused by the integration not being able to fetch new data, because a data set always ends at 00:00.

mampfes commented 1 year ago

I uploaded a debug version in branch debug_stopiteration. Could you please try this out and post the log output one the error occured again?

The only change is the following, so you can also edit the one file manually:

diff --git a/custom_components/epex_spot/__init__.py b/custom_components/epex_spot/__init__.py
index 179f19a..0b04873 100644
--- a/custom_components/epex_spot/__init__.py
+++ b/custom_components/epex_spot/__init__.py
@@ -91,9 +91,12 @@ class SourceDecorator:
         now = dt.now()

         # find current entry in marketdata list
-        self._marketdata_now = next(
-            filter(lambda e: e.start_time <= now and e.end_time > now, self.marketdata)
-        )
+        try:
+            self._marketdata_now = next(
+                filter(lambda e: e.start_time <= now and e.end_time > now, self.marketdata)
+            )
+        except StopIteration as e:
+            _LOGGER.error(f"now={now}, marketdata={self.marketdata}")

         # get list of entries for today
         start_of_day = now.replace(hour=0, minute=0, second=0, microsecond=0)
fblaese commented 1 year ago

I have installed the debug version, thanks! I will report as soon as this issue appears again.

fblaese commented 1 year ago

The issue did appear again tonight. This time - due to the catched exception - the timer is still running.

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.epex_spot
Source: custom_components/epex_spot/__init__.py:192
Integration: EPEX Spot (documentation, issues)
First occurred: 00:00:00 (19 occurrences)
Last logged: 18:00:00

now=2023-03-16 14:00:00.347874+01:00, marketdata=[Marketprice(start: 2023-03-13T23:00:00+00:00, end: 2023-03-14T00:00:00+00:00, marketprice: 9.57 EUR/MWh), Marketprice(start: 2023-03-14T00:00:00+00:00, end: 2023-03-14T01:00:00+00:00, marketprice: 4.46 EUR/MWh), Marketprice(start: 2023-03-14T01:00:00+00:00, end: 2023-03-14T02:00:00+00:00, marketprice: 4.0 EUR/MWh), Marketprice(start: 2023-03-14T02:00:00+00:00, end: 2023-03-14T03:00:00+00:00, marketprice: 1.84 EUR/MWh), Marketprice(start: 2023-03-14T03:00:00+00:00, end: 2023-03-14T04:00:00+00:00, marketprice: 3.5 EUR/MWh), Marketprice(start: 2023-03-14T04:00:00+00:00, end: 2023-03-14T05:00:00+00:00, marketprice: 9.03 EUR/MWh), Marketprice(start: 2023-03-14T05:00:00+00:00, end: 2023-03-14T06:00:00+00:00, marketprice: 42.6 EUR/MWh), Marketprice(start: 2023-03-14T06:00:00+00:00, end: 2023-03-14T07:00:00+00:00, marketprice: 90.42 EUR/MWh), Marketprice(start: 2023-03-14T07:00:00+00:00, end: 2023-03-14T08:00:00+00:00, marketprice: 94.84 EUR/MWh), Marketprice(start: 2023-03-14T08:00:00+00:00, end: 2023-03-14T09:00:00+00:00, marketprice: 83.37 EUR/MWh), Marketprice(start: 2023-03-14T09:00:00+00:00, end: 2023-03-14T10:00:00+00:00, marketprice: 73.78 EUR/MWh), Marketprice(start: 2023-03-14T10:00:00+00:00, end: 2023-03-14T11:00:00+00:00, marketprice: 60.94 EUR/MWh), Marketprice(start: 2023-03-14T11:00:00+00:00, end: 2023-03-14T12:00:00+00:00, marketprice: 53.26 EUR/MWh), Marketprice(start: 2023-03-14T12:00:00+00:00, end: 2023-03-14T13:00:00+00:00, marketprice: 61.07 EUR/MWh), Marketprice(start: 2023-03-14T13:00:00+00:00, end: 2023-03-14T14:00:00+00:00, marketprice: 65.45 EUR/MWh), Marketprice(start: 2023-03-14T14:00:00+00:00, end: 2023-03-14T15:00:00+00:00, marketprice: 59.82 EUR/MWh), Marketprice(start: 2023-03-14T15:00:00+00:00, end: 2023-03-14T16:00:00+00:00, marketprice: 78.83 EUR/MWh), Marketprice(start: 2023-03-14T16:00:00+00:00, end: 2023-03-14T17:00:00+00:00, marketprice: 122.85 EUR/MWh), Marketprice(start: 2023-03-14T17:00:00+00:00, end: 2023-03-14T18:00:00+00:00, marketprice: 144.86 EUR/MWh), Marketprice(start: 2023-03-14T18:00:00+00:00, end: 2023-03-14T19:00:00+00:00, marketprice: 152.21 EUR/MWh), Marketprice(start: 2023-03-14T19:00:00+00:00, end: 2023-03-14T20:00:00+00:00, marketprice: 140.93 EUR/MWh), Marketprice(start: 2023-03-14T20:00:00+00:00, end: 2023-03-14T21:00:00+00:00, marketprice: 129.72 EUR/MWh), Marketprice(start: 2023-03-14T21:00:00+00:00, end: 2023-03-14T22:00:00+00:00, marketprice: 122.84 EUR/MWh), Marketprice(start: 2023-03-14T22:00:00+00:00, end: 2023-03-14T23:00:00+00:00, marketprice: 105.32 EUR/MWh), Marketprice(start: 2023-03-14T23:00:00+00:00, end: 2023-03-15T00:00:00+00:00, marketprice: 105.64 EUR/MWh), Marketprice(start: 2023-03-15T00:00:00+00:00, end: 2023-03-15T01:00:00+00:00, marketprice: 105.8 EUR/MWh), Marketprice(start: 2023-03-15T01:00:00+00:00, end: 2023-03-15T02:00:00+00:00, marketprice: 103.97 EUR/MWh), Marketprice(start: 2023-03-15T02:00:00+00:00, end: 2023-03-15T03:00:00+00:00, marketprice: 105.23 EUR/MWh), Marketprice(start: 2023-03-15T03:00:00+00:00, end: 2023-03-15T04:00:00+00:00, marketprice: 105.67 EUR/MWh), Marketprice(start: 2023-03-15T04:00:00+00:00, end: 2023-03-15T05:00:00+00:00, marketprice: 121.43 EUR/MWh), Marketprice(start: 2023-03-15T05:00:00+00:00, end: 2023-03-15T06:00:00+00:00, marketprice: 143.47 EUR/MWh), Marketprice(start: 2023-03-15T06:00:00+00:00, end: 2023-03-15T07:00:00+00:00, marketprice: 160.0 EUR/MWh), Marketprice(start: 2023-03-15T07:00:00+00:00, end: 2023-03-15T08:00:00+00:00, marketprice: 159.58 EUR/MWh), Marketprice(start: 2023-03-15T08:00:00+00:00, end: 2023-03-15T09:00:00+00:00, marketprice: 129.99 EUR/MWh), Marketprice(start: 2023-03-15T09:00:00+00:00, end: 2023-03-15T10:00:00+00:00, marketprice: 114.15 EUR/MWh), Marketprice(start: 2023-03-15T10:00:00+00:00, end: 2023-03-15T11:00:00+00:00, marketprice: 97.0 EUR/MWh), Marketprice(start: 2023-03-15T11:00:00+00:00, end: 2023-03-15T12:00:00+00:00, marketprice: 90.85 EUR/MWh), Marketprice(start: 2023-03-15T12:00:00+00:00, end: 2023-03-15T13:00:00+00:00, marketprice: 90.84 EUR/MWh), Marketprice(start: 2023-03-15T13:00:00+00:00, end: 2023-03-15T14:00:00+00:00, marketprice: 96.25 EUR/MWh), Marketprice(start: 2023-03-15T14:00:00+00:00, end: 2023-03-15T15:00:00+00:00, marketprice: 102.83 EUR/MWh), Marketprice(start: 2023-03-15T15:00:00+00:00, end: 2023-03-15T16:00:00+00:00, marketprice: 116.99 EUR/MWh), Marketprice(start: 2023-03-15T16:00:00+00:00, end: 2023-03-15T17:00:00+00:00, marketprice: 150.8 EUR/MWh), Marketprice(start: 2023-03-15T17:00:00+00:00, end: 2023-03-15T18:00:00+00:00, marketprice: 182.3 EUR/MWh), Marketprice(start: 2023-03-15T18:00:00+00:00, end: 2023-03-15T19:00:00+00:00, marketprice: 199.28 EUR/MWh), Marketprice(start: 2023-03-15T19:00:00+00:00, end: 2023-03-15T20:00:00+00:00, marketprice: 168.98 EUR/MWh), Marketprice(start: 2023-03-15T20:00:00+00:00, end: 2023-03-15T21:00:00+00:00, marketprice: 147.36 EUR/MWh), Marketprice(start: 2023-03-15T21:00:00+00:00, end: 2023-03-15T22:00:00+00:00, marketprice: 139.35 EUR/MWh), Marketprice(start: 2023-03-15T22:00:00+00:00, end: 2023-03-15T23:00:00+00:00, marketprice: 127.63 EUR/MWh)]
now=2023-03-16 15:00:00.326703+01:00, marketdata=[Marketprice(start: 2023-03-13T23:00:00+00:00, end: 2023-03-14T00:00:00+00:00, marketprice: 9.57 EUR/MWh), Marketprice(start: 2023-03-14T00:00:00+00:00, end: 2023-03-14T01:00:00+00:00, marketprice: 4.46 EUR/MWh), Marketprice(start: 2023-03-14T01:00:00+00:00, end: 2023-03-14T02:00:00+00:00, marketprice: 4.0 EUR/MWh), Marketprice(start: 2023-03-14T02:00:00+00:00, end: 2023-03-14T03:00:00+00:00, marketprice: 1.84 EUR/MWh), Marketprice(start: 2023-03-14T03:00:00+00:00, end: 2023-03-14T04:00:00+00:00, marketprice: 3.5 EUR/MWh), Marketprice(start: 2023-03-14T04:00:00+00:00, end: 2023-03-14T05:00:00+00:00, marketprice: 9.03 EUR/MWh), Marketprice(start: 2023-03-14T05:00:00+00:00, end: 2023-03-14T06:00:00+00:00, marketprice: 42.6 EUR/MWh), Marketprice(start: 2023-03-14T06:00:00+00:00, end: 2023-03-14T07:00:00+00:00, marketprice: 90.42 EUR/MWh), Marketprice(start: 2023-03-14T07:00:00+00:00, end: 2023-03-14T08:00:00+00:00, marketprice: 94.84 EUR/MWh), Marketprice(start: 2023-03-14T08:00:00+00:00, end: 2023-03-14T09:00:00+00:00, marketprice: 83.37 EUR/MWh), Marketprice(start: 2023-03-14T09:00:00+00:00, end: 2023-03-14T10:00:00+00:00, marketprice: 73.78 EUR/MWh), Marketprice(start: 2023-03-14T10:00:00+00:00, end: 2023-03-14T11:00:00+00:00, marketprice: 60.94 EUR/MWh), Marketprice(start: 2023-03-14T11:00:00+00:00, end: 2023-03-14T12:00:00+00:00, marketprice: 53.26 EUR/MWh), Marketprice(start: 2023-03-14T12:00:00+00:00, end: 2023-03-14T13:00:00+00:00, marketprice: 61.07 EUR/MWh), Marketprice(start: 2023-03-14T13:00:00+00:00, end: 2023-03-14T14:00:00+00:00, marketprice: 65.45 EUR/MWh), Marketprice(start: 2023-03-14T14:00:00+00:00, end: 2023-03-14T15:00:00+00:00, marketprice: 59.82 EUR/MWh), Marketprice(start: 2023-03-14T15:00:00+00:00, end: 2023-03-14T16:00:00+00:00, marketprice: 78.83 EUR/MWh), Marketprice(start: 2023-03-14T16:00:00+00:00, end: 2023-03-14T17:00:00+00:00, marketprice: 122.85 EUR/MWh), Marketprice(start: 2023-03-14T17:00:00+00:00, end: 2023-03-14T18:00:00+00:00, marketprice: 144.86 EUR/MWh), Marketprice(start: 2023-03-14T18:00:00+00:00, end: 2023-03-14T19:00:00+00:00, marketprice: 152.21 EUR/MWh), Marketprice(start: 2023-03-14T19:00:00+00:00, end: 2023-03-14T20:00:00+00:00, marketprice: 140.93 EUR/MWh), Marketprice(start: 2023-03-14T20:00:00+00:00, end: 2023-03-14T21:00:00+00:00, marketprice: 129.72 EUR/MWh), Marketprice(start: 2023-03-14T21:00:00+00:00, end: 2023-03-14T22:00:00+00:00, marketprice: 122.84 EUR/MWh), Marketprice(start: 2023-03-14T22:00:00+00:00, end: 2023-03-14T23:00:00+00:00, marketprice: 105.32 EUR/MWh), Marketprice(start: 2023-03-14T23:00:00+00:00, end: 2023-03-15T00:00:00+00:00, marketprice: 105.64 EUR/MWh), Marketprice(start: 2023-03-15T00:00:00+00:00, end: 2023-03-15T01:00:00+00:00, marketprice: 105.8 EUR/MWh), Marketprice(start: 2023-03-15T01:00:00+00:00, end: 2023-03-15T02:00:00+00:00, marketprice: 103.97 EUR/MWh), Marketprice(start: 2023-03-15T02:00:00+00:00, end: 2023-03-15T03:00:00+00:00, marketprice: 105.23 EUR/MWh), Marketprice(start: 2023-03-15T03:00:00+00:00, end: 2023-03-15T04:00:00+00:00, marketprice: 105.67 EUR/MWh), Marketprice(start: 2023-03-15T04:00:00+00:00, end: 2023-03-15T05:00:00+00:00, marketprice: 121.43 EUR/MWh), Marketprice(start: 2023-03-15T05:00:00+00:00, end: 2023-03-15T06:00:00+00:00, marketprice: 143.47 EUR/MWh), Marketprice(start: 2023-03-15T06:00:00+00:00, end: 2023-03-15T07:00:00+00:00, marketprice: 160.0 EUR/MWh), Marketprice(start: 2023-03-15T07:00:00+00:00, end: 2023-03-15T08:00:00+00:00, marketprice: 159.58 EUR/MWh), Marketprice(start: 2023-03-15T08:00:00+00:00, end: 2023-03-15T09:00:00+00:00, marketprice: 129.99 EUR/MWh), Marketprice(start: 2023-03-15T09:00:00+00:00, end: 2023-03-15T10:00:00+00:00, marketprice: 114.15 EUR/MWh), Marketprice(start: 2023-03-15T10:00:00+00:00, end: 2023-03-15T11:00:00+00:00, marketprice: 97.0 EUR/MWh), Marketprice(start: 2023-03-15T11:00:00+00:00, end: 2023-03-15T12:00:00+00:00, marketprice: 90.85 EUR/MWh), Marketprice(start: 2023-03-15T12:00:00+00:00, end: 2023-03-15T13:00:00+00:00, marketprice: 90.84 EUR/MWh), Marketprice(start: 2023-03-15T13:00:00+00:00, end: 2023-03-15T14:00:00+00:00, marketprice: 96.25 EUR/MWh), Marketprice(start: 2023-03-15T14:00:00+00:00, end: 2023-03-15T15:00:00+00:00, marketprice: 102.83 EUR/MWh), Marketprice(start: 2023-03-15T15:00:00+00:00, end: 2023-03-15T16:00:00+00:00, marketprice: 116.99 EUR/MWh), Marketprice(start: 2023-03-15T16:00:00+00:00, end: 2023-03-15T17:00:00+00:00, marketprice: 150.8 EUR/MWh), Marketprice(start: 2023-03-15T17:00:00+00:00, end: 2023-03-15T18:00:00+00:00, marketprice: 182.3 EUR/MWh), Marketprice(start: 2023-03-15T18:00:00+00:00, end: 2023-03-15T19:00:00+00:00, marketprice: 199.28 EUR/MWh), Marketprice(start: 2023-03-15T19:00:00+00:00, end: 2023-03-15T20:00:00+00:00, marketprice: 168.98 EUR/MWh), Marketprice(start: 2023-03-15T20:00:00+00:00, end: 2023-03-15T21:00:00+00:00, marketprice: 147.36 EUR/MWh), Marketprice(start: 2023-03-15T21:00:00+00:00, end: 2023-03-15T22:00:00+00:00, marketprice: 139.35 EUR/MWh), Marketprice(start: 2023-03-15T22:00:00+00:00, end: 2023-03-15T23:00:00+00:00, marketprice: 127.63 EUR/MWh)]
now=2023-03-16 16:00:00.329179+01:00, marketdata=[Marketprice(start: 2023-03-13T23:00:00+00:00, end: 2023-03-14T00:00:00+00:00, marketprice: 9.57 EUR/MWh), Marketprice(start: 2023-03-14T00:00:00+00:00, end: 2023-03-14T01:00:00+00:00, marketprice: 4.46 EUR/MWh), Marketprice(start: 2023-03-14T01:00:00+00:00, end: 2023-03-14T02:00:00+00:00, marketprice: 4.0 EUR/MWh), Marketprice(start: 2023-03-14T02:00:00+00:00, end: 2023-03-14T03:00:00+00:00, marketprice: 1.84 EUR/MWh), Marketprice(start: 2023-03-14T03:00:00+00:00, end: 2023-03-14T04:00:00+00:00, marketprice: 3.5 EUR/MWh), Marketprice(start: 2023-03-14T04:00:00+00:00, end: 2023-03-14T05:00:00+00:00, marketprice: 9.03 EUR/MWh), Marketprice(start: 2023-03-14T05:00:00+00:00, end: 2023-03-14T06:00:00+00:00, marketprice: 42.6 EUR/MWh), Marketprice(start: 2023-03-14T06:00:00+00:00, end: 2023-03-14T07:00:00+00:00, marketprice: 90.42 EUR/MWh), Marketprice(start: 2023-03-14T07:00:00+00:00, end: 2023-03-14T08:00:00+00:00, marketprice: 94.84 EUR/MWh), Marketprice(start: 2023-03-14T08:00:00+00:00, end: 2023-03-14T09:00:00+00:00, marketprice: 83.37 EUR/MWh), Marketprice(start: 2023-03-14T09:00:00+00:00, end: 2023-03-14T10:00:00+00:00, marketprice: 73.78 EUR/MWh), Marketprice(start: 2023-03-14T10:00:00+00:00, end: 2023-03-14T11:00:00+00:00, marketprice: 60.94 EUR/MWh), Marketprice(start: 2023-03-14T11:00:00+00:00, end: 2023-03-14T12:00:00+00:00, marketprice: 53.26 EUR/MWh), Marketprice(start: 2023-03-14T12:00:00+00:00, end: 2023-03-14T13:00:00+00:00, marketprice: 61.07 EUR/MWh), Marketprice(start: 2023-03-14T13:00:00+00:00, end: 2023-03-14T14:00:00+00:00, marketprice: 65.45 EUR/MWh), Marketprice(start: 2023-03-14T14:00:00+00:00, end: 2023-03-14T15:00:00+00:00, marketprice: 59.82 EUR/MWh), Marketprice(start: 2023-03-14T15:00:00+00:00, end: 2023-03-14T16:00:00+00:00, marketprice: 78.83 EUR/MWh), Marketprice(start: 2023-03-14T16:00:00+00:00, end: 2023-03-14T17:00:00+00:00, marketprice: 122.85 EUR/MWh), Marketprice(start: 2023-03-14T17:00:00+00:00, end: 2023-03-14T18:00:00+00:00, marketprice: 144.86 EUR/MWh), Marketprice(start: 2023-03-14T18:00:00+00:00, end: 2023-03-14T19:00:00+00:00, marketprice: 152.21 EUR/MWh), Marketprice(start: 2023-03-14T19:00:00+00:00, end: 2023-03-14T20:00:00+00:00, marketprice: 140.93 EUR/MWh), Marketprice(start: 2023-03-14T20:00:00+00:00, end: 2023-03-14T21:00:00+00:00, marketprice: 129.72 EUR/MWh), Marketprice(start: 2023-03-14T21:00:00+00:00, end: 2023-03-14T22:00:00+00:00, marketprice: 122.84 EUR/MWh), Marketprice(start: 2023-03-14T22:00:00+00:00, end: 2023-03-14T23:00:00+00:00, marketprice: 105.32 EUR/MWh), Marketprice(start: 2023-03-14T23:00:00+00:00, end: 2023-03-15T00:00:00+00:00, marketprice: 105.64 EUR/MWh), Marketprice(start: 2023-03-15T00:00:00+00:00, end: 2023-03-15T01:00:00+00:00, marketprice: 105.8 EUR/MWh), Marketprice(start: 2023-03-15T01:00:00+00:00, end: 2023-03-15T02:00:00+00:00, marketprice: 103.97 EUR/MWh), Marketprice(start: 2023-03-15T02:00:00+00:00, end: 2023-03-15T03:00:00+00:00, marketprice: 105.23 EUR/MWh), Marketprice(start: 2023-03-15T03:00:00+00:00, end: 2023-03-15T04:00:00+00:00, marketprice: 105.67 EUR/MWh), Marketprice(start: 2023-03-15T04:00:00+00:00, end: 2023-03-15T05:00:00+00:00, marketprice: 121.43 EUR/MWh), Marketprice(start: 2023-03-15T05:00:00+00:00, end: 2023-03-15T06:00:00+00:00, marketprice: 143.47 EUR/MWh), Marketprice(start: 2023-03-15T06:00:00+00:00, end: 2023-03-15T07:00:00+00:00, marketprice: 160.0 EUR/MWh), Marketprice(start: 2023-03-15T07:00:00+00:00, end: 2023-03-15T08:00:00+00:00, marketprice: 159.58 EUR/MWh), Marketprice(start: 2023-03-15T08:00:00+00:00, end: 2023-03-15T09:00:00+00:00, marketprice: 129.99 EUR/MWh), Marketprice(start: 2023-03-15T09:00:00+00:00, end: 2023-03-15T10:00:00+00:00, marketprice: 114.15 EUR/MWh), Marketprice(start: 2023-03-15T10:00:00+00:00, end: 2023-03-15T11:00:00+00:00, marketprice: 97.0 EUR/MWh), Marketprice(start: 2023-03-15T11:00:00+00:00, end: 2023-03-15T12:00:00+00:00, marketprice: 90.85 EUR/MWh), Marketprice(start: 2023-03-15T12:00:00+00:00, end: 2023-03-15T13:00:00+00:00, marketprice: 90.84 EUR/MWh), Marketprice(start: 2023-03-15T13:00:00+00:00, end: 2023-03-15T14:00:00+00:00, marketprice: 96.25 EUR/MWh), Marketprice(start: 2023-03-15T14:00:00+00:00, end: 2023-03-15T15:00:00+00:00, marketprice: 102.83 EUR/MWh), Marketprice(start: 2023-03-15T15:00:00+00:00, end: 2023-03-15T16:00:00+00:00, marketprice: 116.99 EUR/MWh), Marketprice(start: 2023-03-15T16:00:00+00:00, end: 2023-03-15T17:00:00+00:00, marketprice: 150.8 EUR/MWh), Marketprice(start: 2023-03-15T17:00:00+00:00, end: 2023-03-15T18:00:00+00:00, marketprice: 182.3 EUR/MWh), Marketprice(start: 2023-03-15T18:00:00+00:00, end: 2023-03-15T19:00:00+00:00, marketprice: 199.28 EUR/MWh), Marketprice(start: 2023-03-15T19:00:00+00:00, end: 2023-03-15T20:00:00+00:00, marketprice: 168.98 EUR/MWh), Marketprice(start: 2023-03-15T20:00:00+00:00, end: 2023-03-15T21:00:00+00:00, marketprice: 147.36 EUR/MWh), Marketprice(start: 2023-03-15T21:00:00+00:00, end: 2023-03-15T22:00:00+00:00, marketprice: 139.35 EUR/MWh), Marketprice(start: 2023-03-15T22:00:00+00:00, end: 2023-03-15T23:00:00+00:00, marketprice: 127.63 EUR/MWh)]
now=2023-03-16 17:00:00.327059+01:00, marketdata=[Marketprice(start: 2023-03-13T23:00:00+00:00, end: 2023-03-14T00:00:00+00:00, marketprice: 9.57 EUR/MWh), Marketprice(start: 2023-03-14T00:00:00+00:00, end: 2023-03-14T01:00:00+00:00, marketprice: 4.46 EUR/MWh), Marketprice(start: 2023-03-14T01:00:00+00:00, end: 2023-03-14T02:00:00+00:00, marketprice: 4.0 EUR/MWh), Marketprice(start: 2023-03-14T02:00:00+00:00, end: 2023-03-14T03:00:00+00:00, marketprice: 1.84 EUR/MWh), Marketprice(start: 2023-03-14T03:00:00+00:00, end: 2023-03-14T04:00:00+00:00, marketprice: 3.5 EUR/MWh), Marketprice(start: 2023-03-14T04:00:00+00:00, end: 2023-03-14T05:00:00+00:00, marketprice: 9.03 EUR/MWh), Marketprice(start: 2023-03-14T05:00:00+00:00, end: 2023-03-14T06:00:00+00:00, marketprice: 42.6 EUR/MWh), Marketprice(start: 2023-03-14T06:00:00+00:00, end: 2023-03-14T07:00:00+00:00, marketprice: 90.42 EUR/MWh), Marketprice(start: 2023-03-14T07:00:00+00:00, end: 2023-03-14T08:00:00+00:00, marketprice: 94.84 EUR/MWh), Marketprice(start: 2023-03-14T08:00:00+00:00, end: 2023-03-14T09:00:00+00:00, marketprice: 83.37 EUR/MWh), Marketprice(start: 2023-03-14T09:00:00+00:00, end: 2023-03-14T10:00:00+00:00, marketprice: 73.78 EUR/MWh), Marketprice(start: 2023-03-14T10:00:00+00:00, end: 2023-03-14T11:00:00+00:00, marketprice: 60.94 EUR/MWh), Marketprice(start: 2023-03-14T11:00:00+00:00, end: 2023-03-14T12:00:00+00:00, marketprice: 53.26 EUR/MWh), Marketprice(start: 2023-03-14T12:00:00+00:00, end: 2023-03-14T13:00:00+00:00, marketprice: 61.07 EUR/MWh), Marketprice(start: 2023-03-14T13:00:00+00:00, end: 2023-03-14T14:00:00+00:00, marketprice: 65.45 EUR/MWh), Marketprice(start: 2023-03-14T14:00:00+00:00, end: 2023-03-14T15:00:00+00:00, marketprice: 59.82 EUR/MWh), Marketprice(start: 2023-03-14T15:00:00+00:00, end: 2023-03-14T16:00:00+00:00, marketprice: 78.83 EUR/MWh), Marketprice(start: 2023-03-14T16:00:00+00:00, end: 2023-03-14T17:00:00+00:00, marketprice: 122.85 EUR/MWh), Marketprice(start: 2023-03-14T17:00:00+00:00, end: 2023-03-14T18:00:00+00:00, marketprice: 144.86 EUR/MWh), Marketprice(start: 2023-03-14T18:00:00+00:00, end: 2023-03-14T19:00:00+00:00, marketprice: 152.21 EUR/MWh), Marketprice(start: 2023-03-14T19:00:00+00:00, end: 2023-03-14T20:00:00+00:00, marketprice: 140.93 EUR/MWh), Marketprice(start: 2023-03-14T20:00:00+00:00, end: 2023-03-14T21:00:00+00:00, marketprice: 129.72 EUR/MWh), Marketprice(start: 2023-03-14T21:00:00+00:00, end: 2023-03-14T22:00:00+00:00, marketprice: 122.84 EUR/MWh), Marketprice(start: 2023-03-14T22:00:00+00:00, end: 2023-03-14T23:00:00+00:00, marketprice: 105.32 EUR/MWh), Marketprice(start: 2023-03-14T23:00:00+00:00, end: 2023-03-15T00:00:00+00:00, marketprice: 105.64 EUR/MWh), Marketprice(start: 2023-03-15T00:00:00+00:00, end: 2023-03-15T01:00:00+00:00, marketprice: 105.8 EUR/MWh), Marketprice(start: 2023-03-15T01:00:00+00:00, end: 2023-03-15T02:00:00+00:00, marketprice: 103.97 EUR/MWh), Marketprice(start: 2023-03-15T02:00:00+00:00, end: 2023-03-15T03:00:00+00:00, marketprice: 105.23 EUR/MWh), Marketprice(start: 2023-03-15T03:00:00+00:00, end: 2023-03-15T04:00:00+00:00, marketprice: 105.67 EUR/MWh), Marketprice(start: 2023-03-15T04:00:00+00:00, end: 2023-03-15T05:00:00+00:00, marketprice: 121.43 EUR/MWh), Marketprice(start: 2023-03-15T05:00:00+00:00, end: 2023-03-15T06:00:00+00:00, marketprice: 143.47 EUR/MWh), Marketprice(start: 2023-03-15T06:00:00+00:00, end: 2023-03-15T07:00:00+00:00, marketprice: 160.0 EUR/MWh), Marketprice(start: 2023-03-15T07:00:00+00:00, end: 2023-03-15T08:00:00+00:00, marketprice: 159.58 EUR/MWh), Marketprice(start: 2023-03-15T08:00:00+00:00, end: 2023-03-15T09:00:00+00:00, marketprice: 129.99 EUR/MWh), Marketprice(start: 2023-03-15T09:00:00+00:00, end: 2023-03-15T10:00:00+00:00, marketprice: 114.15 EUR/MWh), Marketprice(start: 2023-03-15T10:00:00+00:00, end: 2023-03-15T11:00:00+00:00, marketprice: 97.0 EUR/MWh), Marketprice(start: 2023-03-15T11:00:00+00:00, end: 2023-03-15T12:00:00+00:00, marketprice: 90.85 EUR/MWh), Marketprice(start: 2023-03-15T12:00:00+00:00, end: 2023-03-15T13:00:00+00:00, marketprice: 90.84 EUR/MWh), Marketprice(start: 2023-03-15T13:00:00+00:00, end: 2023-03-15T14:00:00+00:00, marketprice: 96.25 EUR/MWh), Marketprice(start: 2023-03-15T14:00:00+00:00, end: 2023-03-15T15:00:00+00:00, marketprice: 102.83 EUR/MWh), Marketprice(start: 2023-03-15T15:00:00+00:00, end: 2023-03-15T16:00:00+00:00, marketprice: 116.99 EUR/MWh), Marketprice(start: 2023-03-15T16:00:00+00:00, end: 2023-03-15T17:00:00+00:00, marketprice: 150.8 EUR/MWh), Marketprice(start: 2023-03-15T17:00:00+00:00, end: 2023-03-15T18:00:00+00:00, marketprice: 182.3 EUR/MWh), Marketprice(start: 2023-03-15T18:00:00+00:00, end: 2023-03-15T19:00:00+00:00, marketprice: 199.28 EUR/MWh), Marketprice(start: 2023-03-15T19:00:00+00:00, end: 2023-03-15T20:00:00+00:00, marketprice: 168.98 EUR/MWh), Marketprice(start: 2023-03-15T20:00:00+00:00, end: 2023-03-15T21:00:00+00:00, marketprice: 147.36 EUR/MWh), Marketprice(start: 2023-03-15T21:00:00+00:00, end: 2023-03-15T22:00:00+00:00, marketprice: 139.35 EUR/MWh), Marketprice(start: 2023-03-15T22:00:00+00:00, end: 2023-03-15T23:00:00+00:00, marketprice: 127.63 EUR/MWh)]
now=2023-03-16 18:00:00.326660+01:00, marketdata=[Marketprice(start: 2023-03-13T23:00:00+00:00, end: 2023-03-14T00:00:00+00:00, marketprice: 9.57 EUR/MWh), Marketprice(start: 2023-03-14T00:00:00+00:00, end: 2023-03-14T01:00:00+00:00, marketprice: 4.46 EUR/MWh), Marketprice(start: 2023-03-14T01:00:00+00:00, end: 2023-03-14T02:00:00+00:00, marketprice: 4.0 EUR/MWh), Marketprice(start: 2023-03-14T02:00:00+00:00, end: 2023-03-14T03:00:00+00:00, marketprice: 1.84 EUR/MWh), Marketprice(start: 2023-03-14T03:00:00+00:00, end: 2023-03-14T04:00:00+00:00, marketprice: 3.5 EUR/MWh), Marketprice(start: 2023-03-14T04:00:00+00:00, end: 2023-03-14T05:00:00+00:00, marketprice: 9.03 EUR/MWh), Marketprice(start: 2023-03-14T05:00:00+00:00, end: 2023-03-14T06:00:00+00:00, marketprice: 42.6 EUR/MWh), Marketprice(start: 2023-03-14T06:00:00+00:00, end: 2023-03-14T07:00:00+00:00, marketprice: 90.42 EUR/MWh), Marketprice(start: 2023-03-14T07:00:00+00:00, end: 2023-03-14T08:00:00+00:00, marketprice: 94.84 EUR/MWh), Marketprice(start: 2023-03-14T08:00:00+00:00, end: 2023-03-14T09:00:00+00:00, marketprice: 83.37 EUR/MWh), Marketprice(start: 2023-03-14T09:00:00+00:00, end: 2023-03-14T10:00:00+00:00, marketprice: 73.78 EUR/MWh), Marketprice(start: 2023-03-14T10:00:00+00:00, end: 2023-03-14T11:00:00+00:00, marketprice: 60.94 EUR/MWh), Marketprice(start: 2023-03-14T11:00:00+00:00, end: 2023-03-14T12:00:00+00:00, marketprice: 53.26 EUR/MWh), Marketprice(start: 2023-03-14T12:00:00+00:00, end: 2023-03-14T13:00:00+00:00, marketprice: 61.07 EUR/MWh), Marketprice(start: 2023-03-14T13:00:00+00:00, end: 2023-03-14T14:00:00+00:00, marketprice: 65.45 EUR/MWh), Marketprice(start: 2023-03-14T14:00:00+00:00, end: 2023-03-14T15:00:00+00:00, marketprice: 59.82 EUR/MWh), Marketprice(start: 2023-03-14T15:00:00+00:00, end: 2023-03-14T16:00:00+00:00, marketprice: 78.83 EUR/MWh), Marketprice(start: 2023-03-14T16:00:00+00:00, end: 2023-03-14T17:00:00+00:00, marketprice: 122.85 EUR/MWh), Marketprice(start: 2023-03-14T17:00:00+00:00, end: 2023-03-14T18:00:00+00:00, marketprice: 144.86 EUR/MWh), Marketprice(start: 2023-03-14T18:00:00+00:00, end: 2023-03-14T19:00:00+00:00, marketprice: 152.21 EUR/MWh), Marketprice(start: 2023-03-14T19:00:00+00:00, end: 2023-03-14T20:00:00+00:00, marketprice: 140.93 EUR/MWh), Marketprice(start: 2023-03-14T20:00:00+00:00, end: 2023-03-14T21:00:00+00:00, marketprice: 129.72 EUR/MWh), Marketprice(start: 2023-03-14T21:00:00+00:00, end: 2023-03-14T22:00:00+00:00, marketprice: 122.84 EUR/MWh), Marketprice(start: 2023-03-14T22:00:00+00:00, end: 2023-03-14T23:00:00+00:00, marketprice: 105.32 EUR/MWh), Marketprice(start: 2023-03-14T23:00:00+00:00, end: 2023-03-15T00:00:00+00:00, marketprice: 105.64 EUR/MWh), Marketprice(start: 2023-03-15T00:00:00+00:00, end: 2023-03-15T01:00:00+00:00, marketprice: 105.8 EUR/MWh), Marketprice(start: 2023-03-15T01:00:00+00:00, end: 2023-03-15T02:00:00+00:00, marketprice: 103.97 EUR/MWh), Marketprice(start: 2023-03-15T02:00:00+00:00, end: 2023-03-15T03:00:00+00:00, marketprice: 105.23 EUR/MWh), Marketprice(start: 2023-03-15T03:00:00+00:00, end: 2023-03-15T04:00:00+00:00, marketprice: 105.67 EUR/MWh), Marketprice(start: 2023-03-15T04:00:00+00:00, end: 2023-03-15T05:00:00+00:00, marketprice: 121.43 EUR/MWh), Marketprice(start: 2023-03-15T05:00:00+00:00, end: 2023-03-15T06:00:00+00:00, marketprice: 143.47 EUR/MWh), Marketprice(start: 2023-03-15T06:00:00+00:00, end: 2023-03-15T07:00:00+00:00, marketprice: 160.0 EUR/MWh), Marketprice(start: 2023-03-15T07:00:00+00:00, end: 2023-03-15T08:00:00+00:00, marketprice: 159.58 EUR/MWh), Marketprice(start: 2023-03-15T08:00:00+00:00, end: 2023-03-15T09:00:00+00:00, marketprice: 129.99 EUR/MWh), Marketprice(start: 2023-03-15T09:00:00+00:00, end: 2023-03-15T10:00:00+00:00, marketprice: 114.15 EUR/MWh), Marketprice(start: 2023-03-15T10:00:00+00:00, end: 2023-03-15T11:00:00+00:00, marketprice: 97.0 EUR/MWh), Marketprice(start: 2023-03-15T11:00:00+00:00, end: 2023-03-15T12:00:00+00:00, marketprice: 90.85 EUR/MWh), Marketprice(start: 2023-03-15T12:00:00+00:00, end: 2023-03-15T13:00:00+00:00, marketprice: 90.84 EUR/MWh), Marketprice(start: 2023-03-15T13:00:00+00:00, end: 2023-03-15T14:00:00+00:00, marketprice: 96.25 EUR/MWh), Marketprice(start: 2023-03-15T14:00:00+00:00, end: 2023-03-15T15:00:00+00:00, marketprice: 102.83 EUR/MWh), Marketprice(start: 2023-03-15T15:00:00+00:00, end: 2023-03-15T16:00:00+00:00, marketprice: 116.99 EUR/MWh), Marketprice(start: 2023-03-15T16:00:00+00:00, end: 2023-03-15T17:00:00+00:00, marketprice: 150.8 EUR/MWh), Marketprice(start: 2023-03-15T17:00:00+00:00, end: 2023-03-15T18:00:00+00:00, marketprice: 182.3 EUR/MWh), Marketprice(start: 2023-03-15T18:00:00+00:00, end: 2023-03-15T19:00:00+00:00, marketprice: 199.28 EUR/MWh), Marketprice(start: 2023-03-15T19:00:00+00:00, end: 2023-03-15T20:00:00+00:00, marketprice: 168.98 EUR/MWh), Marketprice(start: 2023-03-15T20:00:00+00:00, end: 2023-03-15T21:00:00+00:00, marketprice: 147.36 EUR/MWh), Marketprice(start: 2023-03-15T21:00:00+00:00, end: 2023-03-15T22:00:00+00:00, marketprice: 139.35 EUR/MWh), Marketprice(start: 2023-03-15T22:00:00+00:00, end: 2023-03-15T23:00:00+00:00, marketprice: 127.63 EUR/MWh)]

As already suspected, one of the two data sources (awattar) did not refresh its market data in time, which caused the lambda filter to return an empty result. Strangely enough, the awattar data source still does not contain any up-to-date market data, even ~18 hours later.

I also have this error in the log, but it is probably unrelated as it happend after the first catched exception and only affects the epexspot data source (which is still working fine). It probably was caused by an internet outage today.

First occurred: 11:00:09 (1 occurrences)
Last logged: 11:00:09

fetch failed : HTTPSConnectionPool(host='www.epexspot.com', port=443): Max retries exceeded with url: /en/market-data?market_area=DE-LU&trading_date=2023-03-15&delivery_date=2023-03-16&modality=Auction&sub_modality=DayAhead&product=60&data_mode=table&ajax_form=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7fee1a5a3af0>, 'Connection to www.epexspot.com timed out. (connect timeout=None)'))
mampfes commented 1 year ago

I agree, obviously fetching of the data fails. The interesting question is now whether the fetch stops to return valid data, or if the fetch functions is not called anymore.

Therefore I added some more debug output to the branch. You should see 2 lines for every source (awattar, epexspot) right after the hour.

BTW: As no one else has reported this issue so far, I assume this has s.th. to do with your network infrastructure or system setup. Do you s.th. special?

Puma7 commented 1 year ago

epex

Is it possible that I have the same issue? At point midnight there is no refresh of the numbers anymore. (Exept i restart Homeassistant or i reload the intigration, that works as well)

fblaese commented 1 year ago

After the initial fetch after reboot, the awattar fetch is completely missing from the log. Instead epexspot fetch appears twice every hour after that.

The issue might be related to having both data sources configured at the same time.

BTW: As no one else has reported this issue so far, I assume this has s.th. to do with your network infrastructure or system setup. Do you s.th. special?

Nothing that I would know of.

Puma7 commented 1 year ago

Yes, I have both data sources activated. I now deactivate one and have a look if it works :)