search

electricitymaps / electricitymaps-contrib

The open source repository for Electricity Maps App and data parsers that enables a real-time visualisation of the CO2 emissions of electricity consumption
https://app.electricitymaps.com
GNU Affero General Public License v3.0
3.6k stars 945 forks source link

Russia 🇷🇺 #757

Closed denplis closed 4 years ago

denplis commented 7 years ago

Cumulative production per hour: http://so-ups.ru/index.php?id=ees_gen_consump_plan&no_cache=1

Nuclear production, per few days update? http://www.rosenergoatom.ru/ url for parsing data: http://www.rosenergoatom.ru/data.xml

Capacities: https://minenergo.gov.ru/node/532 From this article, we 've got fossil stations structure: gas - 71% coal - 27.5% oil - 1% other - 0.5%

systemcatch commented 7 years ago

@denplis Thanks for the info & links. It looks like the first link has hourly production for each grid region which is useful. I'm assuming that includes all types of generation.

The nuclear generation data is probably for the whole country using this as a capacity reference.

If we could separate out their hydroelectric generation then I think we could write a parser for the area of Russia covered by the grid.

denplis commented 7 years ago

Production by type, updating per hour: http://br.so-ups.ru/Public/MainPageData/BR/PowerGeneration.aspx

Export / Import data http://br.so-ups.ru/Public/MainPage.aspx

denplis commented 7 years ago

https://github.com/tmrowco/electricitymap/pull/766 parser draft https://github.com/tmrowco/electricitymap/pull/767 capacity update

jarek commented 7 years ago

Look like the data in http://br.so-ups.ru/Public/MainPageData/BR/PowerGeneration.aspx is scheduled and not necessarily real-time? It's 16:16 on 23.09 in Moscow right now and the page is returning data for up to 23.09.2017 23 hour.

denplis commented 7 years ago

@jarek you right, that a schedule for the next day. Real production displayed here but
without the type of energy of production. Anyway, difference between plan and real production isn't so big

corradio commented 7 years ago

There's a decision to be made here. Up to know, we have refrained from using estimated numbers for everything as this is supposed to be a live view. @denplis can you maybe try to write to them to know if they have a live breakdown?

systemcatch commented 7 years ago

If we do use the planned generation there should probably be a note on the country breakdown saying it's estimated. Also in the data returned by the parser we could have key like 'estimated'/'scheduled' set to True/False to allow easier filtering of those datapoints.

corradio commented 7 years ago

I've given this a bit of thought and I think this would add complexity (and "corrupt" our database). Let's stick with measured activity. It would be great to get in touch with them though. Maybe @superlopuh can help with his language skills?

superlopuh commented 7 years ago

I've had a bit of a dig around the website. It seems that the map on the front page is showing both the projected and real data. When you look in the future, only the projected is shown, and the transfer between regions is 0.

It looks like this doesn't quite cover the whole country. While it seems to provide data for multiple regions in Russia, the borders look a bit arbitrary to me, but could still be interesting to have a fine-grained breakdown.

The press release thing on the side mentions "young specialists" participating in some competitions and conferences on energetics, which gives me hope that we can get in touch with people familiar with the system that will be willing to help us out.

In any case, I'll try to play around with their API, and see if we can get what we need from it right now.

superlopuh commented 7 years ago

As a side note, here's the hymn of the company! 💯 http://www.so-ups.ru/index.php?id=hymn

systemcatch commented 7 years ago

@superlopuh Can you post the url of the map? We may be able to see where it's requesting real time data from.

hectorespert commented 7 years ago

Api Documentation: http://br.so-ups.ru/Public/Docs/DocView.aspx?id=20a9cf5c-2ab6-4364-8194-eb2ff5911533 Api WSDL: http://br.so-ups.ru:8090/PublicApi/PublicApiService.svc?wsdl

jarek commented 6 years ago

I took a look at the WSDL API: it appears to have basically the same data as various pages on http://br.so-ups.ru/

I also re-checked and still didn't find actual generation by type on the site or in the API. There is actual-vs-planned data for overall generation and demand, but not for generation by type.

alixunderplatz commented 6 years ago

If you work on Russia anyways, something I remember from issue #321 could be implemented. The exports from Kaliningrad-area (RU-KGD) to Lithuania are almost exclusively based on electricity generated from gas. So all exports to Lithuania have an emission factor of 490 g/kWh.

Here is a wikipedia-list of power plants in the RU-KGD area. The only big active unit is Калининградская ТЭЦ-2 / Kaliningradskaya CHPP-2 with 900 MW. The other units (Wind, Hydro, Oil; <= 20 MW) definitely are irrelevant.

corradio commented 6 years ago

What is the status here? It seems like the conclusion is that we don't have access to measured generation mix? If that's the case, we should close the PR #766

systemcatch commented 6 years ago

I think that PR can be closed.

alixunderplatz commented 6 years ago

I am currently checking whether the "planned" generation data is updated, once the hour is over - and I think it really is! This means we would not necessarily rely on forecast data for the future, but on real-time data.

See the following image: the left half was copied about an hour ago, the right half just now, and the numbers have changed for the 23rd hour (yellow).

image

Maybe keep this open until tomorrow, I will track possible changes!

alixunderplatz commented 6 years ago

By the way, these are the "price zones" 1 (European part and Ural) and 2 (south western part of siberia) for the data. So areas for the most important parts of generation and consumption in Russia are covered.

image

image

The upper half of siberia accounts for less than 2% of consumption according to the second image and is isolated from the rest of the grid.

The other grey zones are not covered by the price zones and therefore no generation data by type is available (but total generation!).

corradio commented 6 years ago

THat's very interesting. Let us know when you have a bit more info on the generation mix data.

alixunderplatz commented 6 years ago

Good morning everyone! I kept watching the numbers on http://br.so-ups.ru/Public/MainPageData/BR/PowerGeneration.aspx What I previously assumed seems to be valid.

Summary of everything we know so far:

I speak Russian, too, so I tried to find some more infos on that site regarding the data in general, but there is nothing really helpful there at a first glance. Just many abbreviations. For example, I tried to figure out what "БС" (BS) in the generation stands for, but I have no idea yet. Probably something thermal, as was already assumed.

Conclusion: I think we can use these numbers as they are solid and represent the largest part of Russia's generation. Only some 6-8 GW in the very far eastern Siberian region, 2 GW in the European north and some hundred MW in Kaliningrad area are missing of ca. 120 to 150 GW available generation (price zones 1+2 combined).

For the beginning, we can use Russia as one region and see how it works. Later on, we can split the regions by the given price zones OR Russia west / Russia east.

jarek commented 6 years ago

Great find! One question that would help with implementation: in your tests, at roughly what point past the hours do the values update? E.g. for 22:00, is the actual value inserted at 22:00:01, 22:05, 22:15?

alixunderplatz commented 6 years ago

@jarek I have tracked it in the last hour.

While refreshing the site during minute XX:51 of the last two hours, the site did not reply. Around minute XX:52:15 it was available again and the generating numbers for the following hours were updated. So in minute XX:53, it is definitely available again. I attached an Excel file here, where I used a markup for what has changed and how (see especially rows 505 to 512 for the latest changes!).

Russia_update.xlsx

So at 14:52 Moscow time, the schedule for the intervals (hours) 15 to 23 was changed. That's why I think Interval 15 (probably from 15:00 to 16:00) represents the "final schedule" for generation for the upcoming hour and not exactly the "real time generation" for a previous hour. But since there are so many generating units, any outage of a unit has a small impact only. The unit outage will possibly be considered in the next hours.

In my opinion, the data is precise enough and can be applied to the map, even though it's a "final schedule".

brunolajoie commented 6 years ago

Awesome work!

I think that near term (h+1) generation forecast is a fair assumption for actual production for a start.

Indeed, Russian Federation an extremely negliegible amount of non-dispatchable (intermittent) capacity.

However, it's a pity that we can't breakdown coal, oil, gas, and biomass from "thermal" generation. We will leave it as "unknown" and input an estimated carbon-intnensity factor for it.

For that purpose, @denplis, would you have more details on how did you find this 71% gas and 29% coal mentioned in #766, for the average thermal power production mix of Russia? Isn't there also a bit of diesel oil and biomass in it ? Please find below the table that we need to complete before we can add RU to the map

RU grid-connected capacity (excl. autoconsumption) MW (source: UN, 2014) TWh for a given year (say, 2016?)
nuclear 25304 ?
hydro 50845 ?
hydro storage 1216 ?
geothermal 78 ?
unknown (coal, gas, oil, biomass) 157592 ?
wind 101 ?
solar 227 ?
     
Estimated unknown?    
coal ? ?
oil ? ?
gas ? ?
biomass 1370 (IRENA 2016)  
alixunderplatz commented 6 years ago

@brunolajoie To keep this alive: I found the requested numbers for the year 2015 at IEA:

https://www.iea.org/statistics/statisticssearch/report/?country=Russia&product=electricityandheat

Category unknown consists of:

Therefore, the "unknown" emission factor amounts to: 22.7% 820 g/kWh + 1.4% 650 g/kWh + 75.8% * 490 g/kWh = 567 g/kWh

image

Hydro and other stuff is given, too, but we have access to the live generation data, therefore I didn't list them.

HansHyde commented 6 years ago

@alixunderplatz Hi Alix, regarding your question about conversions (& errors), this chart you presented with electricity and heat, might just shed light on it. Use your conversion to calculate one from the other, and the other from the one. For example, convert heat to electricity & electricity to heat. Compare you numbers.

They might come out to impossible values, such that energy out far exceeds any energy that could have gone in even with giving coal a 30% efficiency as a test. Here's where the other "often missed" factor comes in. I found it while tracking "ejected energy/heat" against efficiencies of a generator with a fixed input of energy to highlight quantity of CO2 from consumable electricity vs from ejected energy at the generator.

PS... are these the one or two nuclear reactors in the world that the nuclear advocacy groups roll out to demonstrate how NPPs "could" be used for process heat (desal, district or process heating), but the fact remains they rarely ever do? ;) It's not an anti-nuclear question btw.

alixunderplatz commented 6 years ago

@HansHyde the numbers presented in this chart do not necessarily represent, how much heat was decoupled from a power station for district heating or industry or how much thermal energy was put into a turbine as steam or whatever. We do not know anything about the amount of primary energy in the form of coal, gas or oil that was put into these power stations, since this is not given in the chart. They might have as well just burned fossil fuels seperately in a boiler to produce the respective heat. So, let's assume electricity and heat in this chart are not related in any kind.

HansHyde commented 6 years ago

You are unable to convert GWh of electricity produced back into energy content of fuel consumed to generate the electricity using an efficiency of 30% for coal?

alixunderplatz commented 6 years ago

@HansHyde

I am sorry, but, I simply don't understand what you actually want to hear from me. I slowly start to believe I am being trolled by you, seriously - but for now I assume I misunderstood what you wrote somewhere.

Do you want me to divide the "electricity" column by 0.3? that would be the primary energy content that was put into the power plant's combustion chamber ... 158,880 GWh/0.3 = 529,600 GWh of energy content in the coal.

What next??? I don't see how any of this is related or relevant to what I posted in this issue right here.

brunolajoie commented 6 years ago

great job for the "unknown" TWh breakdown up to 100% @alixunderplatz

brunolajoie commented 6 years ago

Feel free to open a PR to update it. Plz add this as as commented text on the source code of the PR so we will be able to finetune it later when we will have better LCA per type of fuel for Russia

"assumes weighted average emission factor based on 2015-TWh production: 22.7% 820 g/kWh (coal) + 1.4% 650 g/kWh (oil) + 75.8% * 490 g/kWh (gas) = 567 g/kWh "

alixunderplatz commented 6 years ago

Generation from renewables wind and solar

http://br.so-ups.ru/Public/MainPageData/VSVGO/PowerVIEByRegions.aspx

This table is showing renewable generation from wind and solar power plants in Russia. Some regions clearly can be assigned to generation from solar:

Объем (МВт) 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Алтайский край 0 0 0 0 0 0 3 7 9 11 10 7 3 0 0 0 0 0 0 0 0 0 0 0
Астраханская область 0 0 0 0 0 0 0 0 2 3 3 3 3 3 2 0 0 0 0 0 0 0 0 0
Волгоградская область 0 0 0 0 0 0 0 0 0 0 1 1 2 1 1 0 0 0 0 0 0 0 0 0
Оренбургская область 0 0 0 0 0 0 0 1 8 23 32 36 33 22 6 1 0 0 0 0 0 0 0 0
Республика Башкортостан 0 0 0 0 0 0 0 1 4 10 13 14 12 8 3 0 0 0 0 0 0 0 0 0
Республика Бурятия 0 0 0 0 0 2 3 4 4 3 2 0 0 0 0 0 0 0 0 0 0 0 0 0
Республика Хакасия 0 0 0 0 0 0 1 1 2 2 1 1 0 0 0 0 0 0 0 0 0 0 0 0
Саратовская область 0 0 0 0 0 0 0 0 1 1 2 2 2 2 1 1 0 0 0 0 0 0 0 0

In Crimea region, it is a mix of wind and solar: Республика Крым | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 25 | 27 | 52 | 74 | 100 | 111 | 100 | 68 | 36 | 29 | 24 | 22 | 22 | 22 | 22 | 22 | 22

In Ulyanovsk region, it might be wind (they plan to cover 30% of demand by 2024 from wind): Ульяновская область | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4 | 4

Here is the total amount across Russia from the table above: total in MW | 29 | 29 | 29 | 29 | 29 | 31 | 36 | 43 | 61 | 109 | 142 | 168 | 170 | 140 | 85 | 42 | 33 | 28 | 26 | 26 | 26 | 26 | 26 | 26

The range from ca. 25 MW up to 170 MW might be a drop in the ocean, but lets keep this info in mind. If solar grows even more in the future, it will be really interesting to see, because of the huge east-west distance and timezones.

alixunderplatz commented 6 years ago

Generation and consumption from hydro pumped storage

note: I edited this a few times until I got it really, really right ;)

http://br.so-ups.ru/Public/MainPageData/PDG/PowerTypeDS.aspx select the box that starts with "1" and only "ГАЭС" and you'll see something like this:

image

Currently, there are two large operating pumped storages in Russia with ~1360 MW, a bigger one with 1200 MW in the Moscow region and a smaller one with 140 MW in the south west near the Black Sea (source 140 MW. There is a third smaller one also in the south west (with only ~16 MW).

So we can take the data for the generation and consumption from pumped storage in the csv or xml or the table itself to fill a seperate pumped storage category with data. :)

alixunderplatz commented 6 years ago

Additional info on Russia:

I had to install Silverlight and use Internet Explorer (👍) to access another map from the Russian site to see data for Kaliningrad area (RU-KGD) which is interconnected with Lithuania only:

You go here http://br.so-ups.ru/Public/MainPage.aspx and klick on "Карта БР". Then you klick on Kaliningrad area two times

image

I don't know if that data is accessible, since I had to klick on the map two times to go there. However, the displayed values are:

As already mentioned, generation from RU-KGD is practically 100 % from natural gas. Couldn't we use the "RU-KDG --> LT" exchange from ENTSO-E to create a hidden "dummy generation" for the export with CO2 intensity of natural gas, which is taken into account for imports in Lithuania? This would improve accuracy of LT CO2 mix.

systemcatch commented 6 years ago

@alixunderplatz I'm hopeful that the data is already loaded onto the page before clicking on the "Карта БР" tab. I'll try to get IE working :scream: and see if I can find more info in the page source or dev tools. If we can get the data I think this can be done assuming we are careful with any assumptions made.

alixunderplatz commented 6 years ago

@systemcatch thanks for investigating ;) I can write a brief summary if needed so everything we know and need is in a more compact form. this thread got quite long.

P.S. If you need the installer of IE, I might ship a CD to you ;-P

corradio commented 6 years ago

Here's a list of things left to do (feel free to update my list):

Hyric89 commented 6 years ago

Say, the interconnection from Kaliningrad to Lithuania still registers as Unknown gCO2eq/kWh.

alixunderplatz commented 6 years ago

@Hyric89 this is because Kaliningrad is not included in the reported data for the two Russian price zones that are used at the moment! Select the export arrow and you will see it says "RU-KGD". That's why there is no CO2eq yet! Seperating Kaliningrad is on the list and will be fixed.

alixunderplatz commented 6 years ago

Update for the To-Do-list of @corradio:

alixunderplatz commented 6 years ago

Update of the To-Do-list

jarek commented 6 years ago

@alixunderplatz you should be now able to provide RU-KGD export intensity in https://github.com/tmrowco/electricitymap/blob/master/config/co2eq_parameters.js#L98 (if you do, maybe make a comment to revisit this in a few years when the announced Primorskaya coal plant opens)

alixunderplatz commented 6 years ago

@jarek I'll see if I can find some fresh data for RU-KGD and will submit a PR then. Will include a comment to keep the coalPP (😞 *sigh) in mind

corradio commented 6 years ago

@alixunderplatz is there anything else left to do with RU? Are we able to split it up further? Maybe we should have a "parser to be implemented" label to clearly mark issues where found data, and validated its accuracy, and just need to add it to the map?

alixunderplatz commented 6 years ago

@corradio yes! This should not be too complicated.

The split into two regions:

These should be the adm_1 codes for Bidding zone 2 (discussed with @systemcatch on Slack): RUS-2603 (only the southern tip, but this huuuge zone covers these large islands in the north) RUS-2167 RUS-2397 RUS-2402 RUS-2401 RUS-2403 RUS-2400 RUS-2399 RUS-2403 RUS-2397

In addition, exchanges between zone 1 <-> zone 2 themselves plus zone 1 <-> Kazakhstan and zone 2 <-> Kazakhstan must be implemented and the others assigned to their respective zones. That's it.

The "Far East" region (not in price zones and generation mix) has a total generation figure on the frontpage that could be assigned to "unknown" just for fun (between 3 to 5 GW with unknown shares). Hadn't looked into their generation structure yet.

Summary of possible zones and exchanges:

image

image

alixunderplatz commented 6 years ago

Before I forget: Would be cool to see the Oblast-level (federal state level) outlines of the federal regions within the bidding zones, just like in the U.S. ;-)

jarek commented 6 years ago

Hello,

I've noticed that the zones RU-1 and RU-2 don't have names in the locale files, and so show blank in the zones list:

image

image

What should they be properly named? Is it fine to name them "European Russia (Russia)" and "Western Siberia (Russia)"? What should the names in Russian be? I'm not familiar with the sensitivities and don't want to mis-name things.

alixunderplatz commented 6 years ago

On the website of INTER RAO, it says: https://www.interrao.ru/en/activity/traiding/tradesb.php

For the purpose of electricity price control, the whole country is split into:

the first pricing zone (European Russia and Ural) the second pricing zone (Siberia) three non-pricing zones (Kaliningrad Oblast, Far East, Arkhangelsk Oblast and Komi Republic).

What about choosing these "official" names?

In Russian, that is "Европейская часть и Урал (Россия)" and "Сибирь (Россия)"

And in German "Europäischer Teil und Ural Region (Russland)" and "Sibirien (Russland)"

alixunderplatz commented 6 years ago

We could add the wholesale electricity price if desired:

http://br.so-ups.ru/Public/MainPageData/BR/IndicatorBR.aspx Select price zones 1 and 2 to access the tables. The left column of the two is the correct one to use:

image

alixunderplatz commented 6 years ago

To make this endless thread even longer: The market data for Russia now features a "renewable" category (ВИЭ). http://br.so-ups.ru/Public/MainPageData/BR/PowerGeneration.aspx

image

I suggest implementing this, although the influence is small at the moment, but it diversifies the mix. :)

For both areas, it clearly is SOLAR generation at the moment, no wind data :) European part & Ural region (price zone 1): image

Siberia (price zone 2): image

Russia total (EЭС): image

systemcatch commented 6 years ago

Hey Alex I can do this if you help me with the translations. I think it's just a matter of updating the mapping for each Russian zone. Can you look at the raw data in the link below and see if you can find the renewable key?

raw data