OpenEnergyPlatform / data-preprocessing

Repository for data formatting, import of data, data and metadata review, and data curation.
GNU Affero General Public License v3.0
10 stars 7 forks source link

Review: tso_nep_scenario-framework_2021-2035 #62

Closed Ludee closed 2 years ago

Ludee commented 4 years ago

Issue description

Contributed and published data of the scenario framework (Szenariorahmen) 2021-2035 of the transmission system operators (TSO).

In ihrem Entwurf des Szenariorahmens 2021-2035 haben die Übertragungsnetz­betreiber vier mögliche Szenarien der wahrscheinlichen Entwicklung des Strom­sektors bis zu den Jahren 2035 und 2040 abgebildet. Dabei wird die Energie­wende mit unterschiedlich starken Ausprägungen der Sektorenkopplung und mit einer unterschiedlichen Netzorientierung von Erzeuger- und Verbraucherverhalten realisiert. Die Szenarien orientieren sich an den aktuell geltenden gesetzlichen Rahmenbedingungen und den energiepolitischen Zielen der Bundesregierung. Dazu gehören die Ausbauziele für erneuerbare Energien des Bundes-Klimaschutz­gesetzes und die Kohlenstoffdioxid-Reduktionsziele des Klima­schutz­programms 2030. Source: https://www.netzausbau.de/bedarfsermittlung/2035_2021/szenariorahmen/de.html

Datasets

Workflow checklist

  1. GitHub

    • [x] I have submitted this issue to have metadata and data review documented (Issue #62 )
    • [x] Create a new review-branch and push OEMetadata to new branch (review/tso-nep-sf#62)
  2. OEP

    • [x] Upload data to the OEP in schema ~model_draft~ (see upload tutorial) uploaded to sandbox due to sensitive data
    • [x] Link URL of data in this issue (model_draft.project_nameofdata)
  3. Start a Review

    • [x] Start a pull request (PR) from review-branch to master
    • [x] Assign a reviewer and get in contact
  4. Reviewer section

    • [x] A reviewer starts working on the issue
    • [x] Review data license
    • [x] A reviewer finished working on this issue (and awarded a badge)
    • [x] Update metadata on table
    • [x] Data moved to its final schema
    • [x] Add OEP tags to table
    • [x] Merge PR and delete review-branch
    • [x] Document final links of metadata and data in issue description
    • [x] Close issue

Metadata and data for review

Here are the links to my data and metadata. Naming follows the pattern model_draft.project_nameofdata: Metadata: tso_nep_scenario_framework_2021_2035_scenario.json tso_nep_scenario_framework_2021_2035_timeseries.json

Data: model_draft.tso_nep_scenario_framework_2021_2035_timeseries

Reviewed and published metadata and data

Final Data location:

https://openenergy-platform.org/dataedit/view/scenario/nep_2035_leistungsverteilung_installierter_energietraeger https://openenergy-platform.org/dataedit/view/scenario/nep_2035_nettostromverbrauch_nach_anwendungsbereichen https://openenergy-platform.org/dataedit/view/scenario/nep_2035_nettostromverbrauch_je_bundesland_und_szenario https://openenergy-platform.org/dataedit/view/scenario/nep_2035_methodik_verteilnetzorientierter_lastgangerstellung https://openenergy-platform.org/dataedit/view/scenario/nep_2035_maximaler_flexibilitaetseinsatz_je_tagesstunde https://openenergy-platform.org/dataedit/view/scenario/nep_2035_installierte_leistungen_je_szenario https://openenergy-platform.org/dataedit/view/scenario/nep_2035_installierte_leistung_szenario_distributed_energy

Ludee commented 4 years ago

The data has been provided as .xlsx file. The new oedatamodel has been released and will be tested.

christian-rli commented 3 years ago

I can look into the metadata upload. Several questions @Ludee :

With that info I can finish the review.

Ludee commented 3 years ago

What schema should the table be moved to? scenario?

Yes

Are there any other tables connected to this issue than tso_nep_scenario_framework_2021_2035_ee_pv_location_v2 ?

This was the provided test table (data structure and format). The complete data is a xlsx with several sheets and tables.

Also, I would strongly advise agains creating a new structure like that and move the metadata to the data-review directory. The Upload script can move into the data-import directory, into a special folder, if you like. Do you agree?

I agree. Feel free to improve the structure of the repo. This is ongoing research how to implement an: open peer review process for open data in a community database.

Ludee commented 3 years ago

Sources extracted from file:

[1] Quelle für Historie: | BMWi Energiedaten Gesamtausgabe https://www.bmwi.de/Redaktion/DE/Artikel/Energie/energiedaten-gesamtausgabe.html Stand: September 2019

[2] Quelle für Historie: | Öko-Insititut "Sektorale Abgrenzung der deutschen Treibhausgasemissionen mit einem Schwerpunkt auf die verbrennungsbedingten CO2-Emissionen" https://www.oeko.de/publikationen/p-details/sektorale-abgrenzung-der-deutschen-treibhausgasemissionen-mit-einem-schwerpunkt-auf-die-verbrennungs/ https://www.oeko.de/publikationen/p-details/sektorale-abgrenzung-der-deutschen-treibhausgasemissionen-mit-einem-schwerpunkt-auf-die-verbrennungs-1/ Stand: Juli 2018

[3] Quelle für Historie: | Angaben AGEE-Stat für historische Daten von 2014 bis 2018 Stand August 2019 "https://www.erneuerbare-energien.de/EE/Navigation/DE/Service/Erneuerbare_Energien_in_Zahlen/Zeitreihen/zeitreihen.html" --> 10.12.2019 Angaben aus Mifri verwendet

[4] Angaben ÜNB für Mantelzahlen 2035 und 2040; für 2030 die Eckpunkte des Klimaschutzprogramms "https://www.bundesregierung.de/resource/blob/975226/1679914/e01d6bd855f09bf05cf7498e06d0a3ff/2019-10-09-klima-massnahmen-data.pdf?download=1"

[5] Quelle: Forschungsinstitut für Energiewirtschaft (FfE) München

[6] Quelle: Übertragungsnetzbetreiber

christian-rli commented 3 years ago

As I found out in a call today, the data for this upload have been published under a CC-BY license on the NEP website. The necessary downloads are the excel tables named Kapitel 2 & Kapitel 4 near the bottom of the page.

mjazimm commented 3 years ago

I uploaded the tables from "Daten Kapitel 2" (Netzwerkentwicklungsplan). You can reach them via the following links.

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_leistungsverteilung_installierter_energietraeger

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_nettostromverbrauch_nach_anwendungsbereichen

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_nettostromverbrauch_je_bundesland_und_szenario

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_methodik_verteilnetzorientierter_lastgangerstellung

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_maximaler_flexibilitaetseinsatz_je_tagesstunde

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_installierte_leistungen_je_szenario

https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_installierte_leistung_szenario_distributed_energy

chrwm commented 2 years ago

Hi @mjazimm many thanks for the uploads. Here are my thoughts on the current version of the metadata:

  1. General comment: Is it intended that the metadata and tables are in German? English would reach a broader audience and is more appropriate in my view.
  2. Description: This should contain a short overview of the data's context. At the moment the user has to access further links and information to understand what the data is about.
  3. https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_leistungsverteilung_installierter_energietraeger description missing → unit missing → percent
  4. General comment: Please state the units in each column in the field "unit"
  5. Context: homepage: Link could lead to the data directly (https://www.netzentwicklungsplan.de/sites/default/files/paragraphs-files/Kapitel_2_Daten_NEP_2035_V2021_2_Entwurf.xlsx) instead of the general homepage. The homepage has 17 documents ready for download and the metadata don't state which file applies. Please change the link or make a clear reference to the correct filename.
  6. https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_installierte_leistung_szenario_distributed_energy spatial is not only Deutschland
  7. https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_nettostromverbrauch_je_bundesland_und_szenario and https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_installierte_leistungen_je_szenario spatial are NUTS1 regions (federal states), not Deutschland
  8. https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_leistungsverteilung_installierter_energietraeger temporal: ReferenceDate: is 2019 not 2020
  9. https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_nettostromverbrauch_nach_anwendungsbereichen temporal: ReferenceDate: is 2018 not 2020
  10. Contributors field is not filled
  11. Short column descriptions are missing
christian-rli commented 2 years ago

My thoughts on the uploaded files. I am making changes as I am commenting in order to bring this issue to an end (order is from general to specific):

  1. It's fine that the table data is in German, because that's true to the original. I agree with @chrwm though, that at least the description should be in English. Once there are ontology links in the metadata this might be less urgent of an issue, but with data in a German original the translated info does add significant value. I added some tags: NEP, Netzentwicklungsplan, Szenariorahmen, RLI, CC-BY-4.0, SIROP And copyright attribution: Netzentwicklungsplan Strom 2035 (2021), zweiter Entwurf | Übertragungsnetzbetreiber (M) CC-BY-4.0. Modifications (M) to account for database conforming structure.

  2. I added the description: "The NEP 2035 presents the conversion and expansion requirements in the German onshore and offshore electricity transmission grid against the background of the legal requirements of the Energy Industry Act (Energiewirtschaftsgesetz). The TSOs are planning, developing and building the grid of the future. With the NEP, they show how the transformation of the generation landscape in Germany and the integration of renewable energies can succeed by 2035 or 2040." Plus extra sentence, what specific table contains.

  3. add unit percent https://openenergy-platform.org/dataedit/view/model_draft/nep_2035_leistungsverteilung_installierter_energietraeger

  4. added units

  5. There is homepage and there is documentation. I'm a little bit torn as to what's what, but the currently used link certainly refers to the homepage, so documentation should reference the specific file. we might need to discuss "documentation" as a term again or at least update the key description. for now i'll use: homepage: https://www.netzentwicklungsplan.de/de/netzentwicklungsplaene/netzentwicklungsplan-2035-2021 documentation https://www.netzentwicklungsplan.de/sites/default/files/paragraphs-files/Kapitel_2_Daten_NEP_2035_V2021_2_Entwurf.xlsx as context for all files

  6. added Luxemburg & Austria

  7. change to NUTS1 regions

  8. change ReferenceDate to 2019

  9. change ReferenceDate to 2018

  10. added contributors

  11. add short description