search

electricitymaps / electricitymaps-contrib

A real-time visualisation of the CO2 emissions of electricity consumption
https://app.electricitymaps.com
GNU Affero General Public License v3.0
3.52k stars 937 forks source link

Fluctuating Austrian electricity production #383

Closed alixunderplatz closed 7 years ago

alixunderplatz commented 7 years ago

There seems to be another problem with Austrian data besides #325 I think both are not related, though.

Take a look at the specific CO2 level. During night-time, it is rather flat, but during the day it sometimes shows strong deviations in 30 minute intervals. Something like this appears in no other county. At the same time, the levels of water, gas and coal seem to be going up and down extremely all the time. This permanent on-and-off would ruin every usual conventional power plant within a few months or days ;-D. The up and down causes the domestic production to drop and to increase very hard, so the emissions are influenced by this, too. image

If you take a look at the actual 15 min data from ENTSO-E, the production is absolutely smooth, especially for gas and coal.

image

I’m pretty sure I found the root of all evil: I searched for related stuff and found issue #67. Hence the question: are you still using data from https://www.apg.at/en/market/Markttransparenz/generation/Erzeugung%20pro%20Typ ??? From time to time, their data obviously is too high (look at the last two bars): image I guess this is the cause for that issue. The data on the APG site is somehow corrected to normal values a few moments later, though.

Please, check the data behind this in your data base. I guess using entso-e data instead of APG data might be an option, or updating the APG data a few minutes later again, since they appear to be normal then.

Greetings, Alex

ThierryOllivero commented 7 years ago

Hello Alex. We are now using ENTSO-E for Austria. And I noticed this sinusoidal behavior for AT.

@corradio : any idea?

corradio commented 7 years ago

In order to double check we would have to extract some data using the datascience functions we made. Also I would check the ENTSOE data on their website to be absolutely sure. @alixunderplatz how did you do the analysis of the ENTSOE data? How did you obtain it? Also, this could be explained by https://github.com/corradio/electricitymap/issues/325? Some of their production is given as a negative "unknown", and we ignore negative values except for pumped hydro storage. @martindaniel4 or @brunolajoie are you able to help here?

alixunderplatz commented 7 years ago

@corradio I copied the 15min data for yesterday and the hours that were available today from a chart at the entsoe website and quickly put it in excel to check it. The "water" graph in my image above is a sum of pumped storage, run of river and water reservoir (without consumption).

It would be interesting to check the entso-e data every minute to see if there is an anomaly similar to the APG data, which is then disappering. But I don't think there is such a thing.

@ThierryOllivero I was really, really sure APG data is used because the two values of the purple (too) high bars for gas for a moment were exactly the same as displayed in the electricitymap. I should have taken a screenshot with the numbers back then to show you.

Anyways: I have just (20:14) downloaded the .csv for the 18th February on APG's site and I could reproduce the issue for the latest two values: image Gas is about 2x higher than all of the prior values, coal is 3x as high as all previous values.

By the way: I am not able to download data from ENTSO-E since I don't have an account there. Could you provide some of their latest data for Austria?

corradio commented 7 years ago

I will take a look at this but because we use ENTSOE data, I'm not sure I understand exactly why this is happening.

The weird thing is that I observe oscillations because "unknown" production is fluctuating between 0 and some value (and hydro also), whereas on ENTSOE I don't see those oscillations [1].

However it could be that ENTSOE corrects some of its past data, which would explain the discrepancy. I will think about a way to investigate this.

https://transparency.entsoe.eu/generation/r2/actualGenerationPerProductionType/show?name=&defaultValue=false&viewType=GRAPH&areaType=CTY&atch=false&datepicker-day-offset-select-dv-date-from_input=D&dateTime.dateTime=27.02.2017+00:00|CET|DAYTIMERANGE&dateTime.endDateTime=27.02.2017+00:00|CET|DAYTIMERANGE&area.values=CTY|10YAT-APG------L!CTY|10YAT-APG------L&productionType.values=B01&productionType.values=B02&productionType.values=B03&productionType.values=B04&productionType.values=B05&productionType.values=B06&productionType.values=B07&productionType.values=B08&productionType.values=B09&productionType.values=B10&productionType.values=B11&productionType.values=B12&productionType.values=B13&productionType.values=B14&productionType.values=B20&productionType.values=B15&productionType.values=B16&productionType.values=B17&productionType.values=B18&productionType.values=B19&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2)

corradio commented 7 years ago

Taken at 08:19 UTC+1

image Values for 07:00 and 07:15 are < 800MW

Taken at 09:12 UTC+1: image Previous values at 07:00 and 07:15 have been corrected.

Conclusion: values of the past are updated. Our parser only grabs the latest value and does not update past values. We need to change that. We need to grab the last 24h of data (or today + yesterday) and update all values.