Fetching data from entso-e server results in flat line

[rhpijnacker]

Since yesterday (Oct 4) fetching data from the server yields a set of equal values, instead of the actual data.

I noticed that also the Home Assistant ENTSO-e Transparency Platform Energy Prices custom component failed and issued a hot fix. Maybe the data format changed?

[rhpijnacker]

Seems to be a problem in the entsoe-py library: https://github.com/EnergieID/entsoe-py/issues/346.

[nhoening]

Thanks for linking the issue! Quite surprising of ENTSOE to do a breaking change with little notice.

Did you have a chance to try the forked version with a potential fix?

[rhpijnacker]

I'm trying to apply https://github.com/EnergieID/entsoe-py/pull/347/files, but I am running into dependency problems now:

worker-1  |   Attempting uninstall: pandas
worker-1  |     Found existing installation: pandas 2.1.4
worker-1  |     Uninstalling pandas-2.1.4:
worker-1  |       Successfully uninstalled pandas-2.1.4
worker-1  | ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
worker-1  | sktime 0.26.0 requires pandas<2.2.0,>=1.1, but you have pandas 2.2.3 which is incompatible.

which (I think) is causing:

[FLEXMEASURES][2024-10-06 07:29:20,329] INFO: Saving 25 beliefs for Sensor Day-ahead prices ...
/usr/var/flexmeasures-instance/plugins/flexmeasures-entsoe/utils.py:245: FutureWarning: 'H' is deprecated and will be removed in a future version. Please use 'h' instead of 'H'.
  (series.index.floor("D") - pd.Timedelta("6H"))
[FLEXMEASURES] Task entsoe-import-day-ahead-prices encountered a problem: ""DataFrame should contain column named 'source'."". More details: {'src_module': '/usr/local/lib/python3.10/dist-packages/timely_beliefs/beliefs/classes.py', 'src_linenr': 910, 'src_func': '__init__', 'src_code': 'raise KeyError("DataFrame should contain column named \'source\'.")'}
[FLEXMEASURES] Reported task entsoe-import-day-ahead-prices status as False

I can report that it fixes the data values, though.

[rhpijnacker]

I'm also experiencing some problems updating to the latest FM server, see #44.

[rhpijnacker]

Quite surprising of ENTSOE to do a breaking change with little notice.

@nhoening It is suggested https://transparency.entsoe.eu/news/widget?id=66f5203d792e84032cbb9b71 might be causing the problem. Late notice indeed.

[fboerman]

hi all! this was a bug in the package. I have pushed a new version 0.6.9 that fixes this!

[Flix6x]

I can verify the fix by upgrading the following on our live cron server:

• entsoe-py from 0.3.8 to 0.6.9
• pandas from 1.5.3 to 2.2.1 ( higher than 2.2.1 is still problematic for timely-beliefs)

@rhpijnacker Feel free to reopen this issue if upgrading did not fix it for you.

[rhpijnacker]

@Flix6x It works on my local installation too.