search

RadekBuczkowski / anomaly_detection

Anomaly detection in multivariate time series with Keras and LSTM layers
MIT License
4 stars 1 forks source link

Data not available anymore. #1

Closed H3X3R closed 3 weeks ago

H3X3R commented 3 weeks ago

I’m trying to get my hands on the hourly consumption data for Denmark, but it looks like the files on the website are down. Is there any other way to access this data, (from 2013 onward) ? Would be a lifesaver if you could give the data. Thanks!

RadekBuczkowski commented 3 weeks ago

Looks like the Nord Pool service no longer publishes the data. The data I used is not accessible via a link as before. But the energy consumption data in Denmark is still free and publicly available. You can download it manually from the Energinet's portal energidataservice.dk (the national electricity transmission operator in Denmark). The data format is slightly different, and the regions DK1 and DK2 are in separate columns. But you can easily compensate for that in Python. There are actually many more interesting columns for data analysis, like wind energy production. But you can of course only use the columns I used. Just select the dates and click download as CSV (it will be over 50MB since 2013).

https://www.energidataservice.dk/tso-electricity/productionconsumptionsettlement

There is a public API available to fetch data from your code as well, but it may be an overkill: https://www.energidataservice.dk/guides/api-guides

RadekBuczkowski commented 3 weeks ago

The consumption you are looking for is in the column: GrossConsumptionMWh

I noticed the values are slightly different than the values in the original file, but they roughly match. Probably the consumption is calculated differently or includes/excludes some losses.

Anyway, since the data is more comprehensive, and includes e.g. wind production. which constitutes for half of the total energy consumption in Denmark, it would be interesting to feed that data to the training as well. :)

H3X3R commented 3 weeks ago

Thanks for replying, I had no hope anyone would reply on a 4 year old repo, but kudos to you <3. The links work, and I have got the data. Thanks for replying as well as solving this data lost problem.