Error doing job: Task exception was never retrieved

[heinemannj]

Traceback (most recent call last):
  File "/config/custom_components/epex_spot/SourceShell.py", line 90, in fetch 
    await self._source.fetch()
  File "/config/custom_components/epex_spot/EPEXSpot/EPEXSpotWeb/__init__.py", line 133, in fetch
    self._marketdata = await self._fetch_day(delivery_date) + await self._fetch_day(
                                                              ^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/epex_spot/EPEXSpot/EPEXSpotWeb/__init__.py", line 138, in _fetch_day
    data = await self._fetch_data(delivery_date)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/epex_spot/EPEXSpot/EPEXSpotWeb/__init__.py", line 188, in _fetch_data
    resp.raise_for_status()
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1059, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 500, message='Internal Server Error', url=URL('https://www.epexspot.com/en/market-data?market_area=DE-LU&trading_date=2024-01-04&delivery_date=2024-01-05&modality=Auction&sub_modality=DayAhead&product=60&data_mode=table&ajax_form=1')

[mampfes]

Looks like a server problem. So nothing we can do on our side. Was this a singular event or do you see it regularly?

[nicojanssens]

+1

In my setup, this error occurs approx 10 times every 24 hours, causing the plugin and the associate helpers to malfunction.

When executing https://www.epexspot.com/en/market-data?market_area=BE&trading_date=2024-01-16&delivery_date=2024-01-17&modality=Auction&sub_modality=DayAhead&product=60&data_mode=table&ajax_form=1 manually, the EPEX server complains about "The uploaded file likely exceeded the maximum file size (400 MB) that this server supports." while the GET request did not contain any payload whatsoever 🤷‍♂️

[mampfes]

the EPEX server complains about "The uploaded file likely exceeded the maximum file size (400 MB) that this server supports." while the GET request did not contain any payload whatsoever 🤷‍♂️ With "executing manually" you probably mean, putting it into a browser, right? This doesn't work because then you do a HTTP-GET request instead of a HTTP-POST request (which needs additional data also). This probably causes the error message.

But regarding the original error, I don't know if there is much we can do. I do not see it on my machine and it sounds like a server error. The only thing that comes into my mind is to ignore this fetch and try it again later.

[mampfes]

Could you help me to try something out: https://github.com/mampfes/ha_epex_spot/blob/310302b4f79d03f7befd76a3f48fe467de563669/custom_components/epex_spot/__init__.py#L108 Could you please please the 58 with a smaller value (50 for example) on your machine and let me know if you still see the issue?

My idea is that there are too many integrations running on the globe which fetch data all at the same time. Maybe the server(s) are overloaded because they get too many requests at the same time.

[heinemannj]

I've done your proposed hack. I will monitor.

[heinemannj]

Additional issue:

Logger: homeassistant
Source: custom_components/epex_spot_sensor/util.py:21
Integration: EPEX Spot Sensor ([documentation](https://github.com/mampfes/ha_epex_spot_sensor), [issues](https://github.com/mampfes/ha_epex_spot_sensor/issues))
First occurred: 13:55:00 (1 occurrences)
Last logged: 13:55:00

Error doing job: Exception in callback async_track_point_in_utc_time.<locals>.run_action(<Job time cha...0x7f752eb2e0>>) at /usr/src/homeassistant/homeassistant/helpers/event.py:1413
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1428, in run_action
    hass.async_run_hass_job(job, utc_point_in_time)
  File "/usr/src/homeassistant/homeassistant/core.py", line 691, in async_run_hass_job
    hassjob.target(*args)
  File "/usr/src/homeassistant/homeassistant/helpers/event.py", line 1703, in pattern_time_change_listener
    hass.async_run_hass_job(job, dt_util.as_local(now) if local else now)
  File "/usr/src/homeassistant/homeassistant/core.py", line 691, in async_run_hass_job
    hassjob.target(*args)
  File "/config/custom_components/epex_spot_sensor/binary_sensor.py", line 174, in async_price_sensor_state_listener
    _on_price_sensor_state_update()
  File "/config/custom_components/epex_spot_sensor/binary_sensor.py", line 167, in _on_price_sensor_state_update
    self._update_state()
  File "/config/custom_components/epex_spot_sensor/binary_sensor.py", line 241, in _update_state
    self._update_state_for_contigous(earliest_start, latest_end, now)
  File "/config/custom_components/epex_spot_sensor/binary_sensor.py", line 294, in _update_state_for_contigous
    marketdata = get_marketdata_from_sensor_attrs(self._sensor_attributes)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/epex_spot_sensor/util.py", line 51, in get_marketdata_from_sensor_attrs
    return [Marketprice(e) for e in data]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/epex_spot_sensor/util.py", line 51, in <listcomp>
    return [Marketprice(e) for e in data]
            ^^^^^^^^^^^^^^
  File "/config/custom_components/epex_spot_sensor/util.py", line 21, in __init__
    raise KeyError()
KeyError

[mampfes]

Any update on this?