Better data source for Germany found

[LunaticManiac]

Hi! You can get a better data source for Germany here https://www.smard.de/home/downloadcenter/download-marktdaten/?downloadAttributes=%7B%22selectedCategory%22:1,%22selectedSubCategory%22:1,%22selectedRegion%22:%2250Hertz%22,%22selectedFileType%22:false,%22from%22:1669849200000,%22to%22:1670799599999%7D

[VIKTORVAV99]

Looks like this source does have all the production information we need but it seems to be lacking exchange data between the different german grids, do you know if said data is also available?

If we can find that data we could most likely split up the current German zone into it's individual grids.

[LunaticManiac]

Hi! After a lot of using Google this website was the best thing I found for 50 Hertz. https://www.50hertz.com/de/Transparenz/Kennzahlen/Netzdaten/Netzbelastung However, the data there is delayed by a day. But: 50 Hertz also shows the current values in their app. So I wonder if you could get better Information out of this. I will take a look at the other grids as soon as I find the time for it.

[LunaticManiac]

Alright, I just thought about it again. Maybe it's not a good idea to grab data that they don't want to have published. What we could do otherwise is getting the grid balance from here: https://www.netztransparenz.de/Daten-zur-Regelenergie/NRV-Saldo/Saldo As 50 Hertz is only connected to TenneT its simple math by removing the imports and exports from the different countries and we should also get the exact value. For the others I would just estimate them the same way as 50 Hertz and assume the one with more provides the ones with less. The data doesn't seem to be publicly available.

This website has a web api btw: https://www.netztransparenz.de/Web-API

[jarek]

Hello,

For previous discussions on splitting Germany by grid regions, see issues https://github.com/electricitymaps/electricitymaps-contrib/issues/1216 and https://github.com/electricitymaps/electricitymaps-contrib/issues/3795

[LunaticManiac]

In case the assumption I made on the exchange inside the German gridsis too much guessing for you, but the way 50Hertz is calculated is okay, I would recommend adding 3 zones instead of 4, where Amperion and TransBW is a combined zone. As the exchange of 50Hertz and TenneT is already known, the only unknown relationship is then TenneT to the combined zone. That way data should still be highly accurate, even though we don't know the relationship between TransBW and Amperion. The resolution of data would still increase much.

[VIKTORVAV99]

Looks like we might still have to wait for the TSOs to review and hopefully adopt one of these proposals unless we want to hack something together ourselves.

Annex I (with maps)

[Spanni26]

That are the sources of energy charts... https://www.energy-charts.info/sources.html?l=de&c=DE Bad thing...energy charts shows nearly 6 GW on 17.12.2022 for Wind, Electricity map shows 0.

[jarek]

That are the sources of energy charts... https://www.energy-charts.info/sources.html?l=de&c=DE Bad thing...energy charts shows nearly 6 GW on 17.12.2022 for Wind, Electricity map shows 0.

That's a separate issue. You can see there's data missing in the Fraunhofer ISE (energy-charts.info) page as well:

Because actual data for all other categories other than wind is missing, right now Electricity Maps is doing its own forecast/estimate, and it's not picking up the wind increase. When data is backfilled into ENTSOE, both Fraunhofer and Electricity Maps will pick it up.

[VIKTORVAV99]

That are the sources of energy charts... https://www.energy-charts.info/sources.html?l=de&c=DE Bad thing...energy charts shows nearly 6 GW on 17.12.2022 for Wind, Electricity map shows 0.

I would suggest opening a new issue regarding Electricity Maps showing 0 wind. Even if the data is estimated it should not show 0 but that's unrelated to this "issue".

[mirkoschaefer]

Hi! You can get a better data source for Germany here https://www.smard.de/home/downloadcenter/download-marktdaten/?downloadAttributes=%7B%22selectedCategory%22:1,%22selectedSubCategory%22:1,%22selectedRegion%22:%2250Hertz%22,%22selectedFileType%22:false,%22from%22:1669849200000,%22to%22:1670799599999%7D

Generally, SMARD.de publishes ENTSO-E data, so there should be no advantage to use their data over the one from the Transparency Page, except that they have a CC BY 4.0 license on it. https://www.smard.de/en/datennutzung

[florianscheidl]

Hey, just found this thread, great to see that you have looked into this. Has anyone had more thoughts on it in the meantime?

[VIKTORVAV99]

Hey @florianscheidl!

I don't think we need to switch to SMARD.de for production data as it has the same data as ENTSO-E right now (smard is the data supplier to ENTSO-E if I'm not wrong, or if it's the other way around). So building a parser for that would be unnecessary. (Same for energy-charts as far as I can tell).

Was a while since I looked but unless they have added exchange data for germanys internal exchanges the case is the same here.

For price it becomes interesting though as it is licensed under a different license than ENTOSO-E but I know there was a plan to have our lawyers look at it first or something like that before implementing it. @madsnedergaard would know more about that.

Was there something specific you had in mind besides these data types?

[mirkoschaefer]

I think there are no new data sources beyond the one mentioned above (ENTSO-E/SMARD). We have implemented regionalization procedures to estimate generation/load values for Federal States. Free version of the conference proceedings article here (preliminary results): https://arxiv.org/abs/2304.02951 Further results and all data: https://zenodo.org/records/7763278

Extending this work, we have implemented a way to estimate flows between Federal states based on an underlying grid model, which can also be applied to flows between TSOs, but it's not published yet. If you have ideas for validation, let me know!

[florianscheidl]

@VIKTORVAV99, thanks, curious to hear what they say. I didn't have anything else in mind at the moment 🙂

@mirkoschaefer, sounds very interesting. Have you considered validating it on other countries with multiple TSOs (like Italy, Norway or Sweden)? Maybe one could treat a collection of smaller countries as one (e.g. Austria, Czech Republic, Hungary, Slovakia)? Happy to hear your thoughts.

[mirkoschaefer]

@florianscheidl Thanks, that's a good idea. Once we have generation and load data for a set of regions (hopefully balanced) and information about the grid connection between the regions, we estimate flows between the regions similar to the method commercial scheduled exchanges are calculated from net imports/exports in a set of regions with flow-based market coupling. So we can do that for the countries in the ENTSO-E region and compare the results with the actual cross-border physical exchanges.

The regionalisation as mentioned above can also be done for other countries in the same way, it's just about collecting and matching data sources (power plant list, matching with per_unit_generation data, load regionalisation, distribution of wind and solar capacities).

[VIKTORVAV99]

Update here: We will be using nordpool price data in the next release, that should take care of the price issue.