review/openfred_weatherdata

Ludee commented 4 years ago

Tables: Note: metadata just differs at the "resources" key.

openfred_flags -> review metadata
openfred_locations -> review metadata
openfred_series -> review metadata
openfred_timespans -> review metadata
openfred_variables -> review metadata

Additional tables:

Keep these tables?:

tables are in schema model_draft

ToDo:

[x] Create metadata @jh-RLI
[x] Update metadata to v1.4.1 @jh-RLI
[x] Check license info @ronny
[x] Create UML @gnn -> @jh-RLI will provide a simple ERM diagram
[x] Move into climate schema @Ludee
[x] Update and release FRED-WAM @christian-rli
[x] validate metadata using omi @jh-RLI
- set "alignment": "horizontal" to "alignment": null to avoid omi validation error.
[x] Add metadata to OEP openfred tables @jh-RLI

jh-RLI commented 4 years ago

@Ludee I think the openfred_powerplants is missing?

jh-RLI commented 4 years ago

Spoke to @Ludee the powerplants table is not part of the weatherdata set and is not to be considered here.

jh-RLI commented 4 years ago

The metadata-strings are created for all listed tables and linked in the issue description. The license information and a short description are still missing.

rpnaut commented 4 years ago

Regarding the openFRED weather data:

They have already an entry in the WDCC:

https://cera-www.dkrz.de/WDCC/ui/cerasearch/entry?acronym=coastDat3_COSMO-CLM_MERRA2

The description and key-words are defined already there:

Name: coastDat3_COSMO-CLM_MERRA2

Project: openFRED (open feed-in time series based on a Renewable Energy Database)

Summary: This is an atmospheric hindcast for Western Europe and the North Atlantic using COSMO-CLM version 5.0 with spectral nudging from 2002-2017. MERRA2 reanalysis data are used as forcing. Additionally transient and monthly aerosol data of the MACv2 climatology are prescribed. The model uses a rotated grid with 566 x 481 grid points and a grid point distance of 0.0625 degrees, the rotated North pole is located at 162.0 W, 39.25 N.
The published data excludes the sponge zone and have 526 x 441 grid points. In rotated coordinates the published simulation data extends from 22.64 W to 10.18 E, 11.2 S to 16.3 N, in geographical coordinates this corresponds to about 12 W to 30 E, 39 N to 60 N.

institution: Helmholtz-Zentrum Geesthacht, Institute of Coastal Research, Germany

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

contact: http://coastmod.hzg.de

originator: Ronny Petrik

crs: EPSG:4326 ContactDr. Ronny Petrik (✉ ronny.petrik@hzg.de )

Keyword(s): COSMO-CLM, hindcast, MERRA2, openSource, regional modelling, renewable energy

rpnaut commented 4 years ago

Bei den Attributen in Temporal stimmt was nicht: End ist 2018 und Resolution ist minimal 15 Minuten und maximal 60 Minuten.

"temporal": { 29 "referenceDate": null, 30 "timeseries": { 31 "start": "2002", 32 "end": "2017", 33 "resolution": "30 min", 34 "alignment": "horizontal", 35 "aggregationType": null 36 } 37 },

rpnaut commented 4 years ago

Die Lizenzen bei den Sources sind alle auch meiner Meinung nach NULL. Public Domain und Vertragsvereinbarungen können als Lizenzen nicht eingestuft werden.

Ludee commented 4 years ago

Project: openFRED (open feed-in time series based on a Renewable Energy Database)

The correct project name is open_FRED

Ludee commented 4 years ago

Die Lizenzen bei den Sources sind alle auch meiner Meinung nach NULL. Public Domain und Vertragsvereinbarungen können als Lizenzen nicht eingestuft werden.

https://github.com/OpenEnergyPlatform/data-preprocessing/blob/review/openfred/data-review/climate.openfred_weatherdata_locations.json#L38-L151

The metadata point sources/licenses means term of use and not the juridical license. Thus if it is licensed under PD/CC0 or other "quasi-licenses" like GeoNutzV it should be documented. It's not important if it is a real license or not.

Ludee commented 4 years ago

Keyword(s): COSMO-CLM, hindcast, MERRA2, openSource, regional modelling, renewable energy

It's not Open Source nor open-source. But never ever openSource. The source code to generate the weather data is not licensed with an open license! It is Open Data!

Ludee commented 4 years ago

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

What is this? Is this an additional source?

rpnaut commented 4 years ago

Okay. At first we adapt the project explanation at WDCC regarding project name and ending of simulation.

At second, we will adapt the keyword "openSource" - huge mistake and never checked by myself.

rpnaut commented 4 years ago

Die Attribution „coastDat3_COSMO-CLM_MERRA2 - open_FRED Weatherdata“ © Helmholtz-Zentrum Geesthacht | ODC-By-1.0"

ist korrekt.

rpnaut commented 4 years ago

I cannot push to the repository. Please try to get this working with my account rpnaut.

As long as this is not fixed, I keep the files on local devices.

rpnaut commented 4 years ago

I have to say that I do not know the difference between all the json-files:

openfred_flags -> review metadata openfred_locations -> review metadata openfred_series -> review metadata openfred_timespans -> review metadata openfred_variables -> review metadata

So, I only start to review the first one.

rpnaut commented 4 years ago

source: int2lm_131101_2.00_clm4, COSMO-CLM5.0_clm14_aerosol_gust (available at DKRZ's LTA WDCC service)

This is the hint on the source code (version of the model).

Ludee commented 3 years ago

Thanks for the reminder @Bachibouzouk This review should be finalized. @gnn @christian-rli can you help?

jh-RLI commented 3 years ago

I have to say that I do not know the difference between all the json-files:

openfred_flags -> review metadata openfred_locations -> review metadata openfred_series -> review metadata openfred_timespans -> review metadata openfred_variables -> review metadata

So, I only start to review the first one.

That's fine, the only difference is the data model description at the key "resources": ....). Each table is described with technical details, there is also a description for each field/column. This might be worth a review. I will use the current state for now.

jh-RLI commented 3 years ago

The OEP metadata validation tool omi does not allow "alignment":"horizontal". Just "left", "middle" or "right" are valid ATM. What would be the appropriate value?

"temporal": {
        .....
        "timeseries": {
            ....
            "alignment": "horizontal",
           .....
        }

jh-RLI commented 3 years ago

There is no "publicationDate": null Edit: Set to date of this issue (2020-02-06)

jh-RLI commented 3 years ago

OEMetadata is now available on each open_FRED table (scroll down to "Metadata Information"): openfred_flags openfred_locations openfred_series openfred_timespans openfred_variables

Also added similar metadata to the additional tables: openfred_feedin_wind_2016 openfred_timeseries_ch

Note:

I had to use my backend DB access to update the Metadata since there are no permissions set on the tables. I'm not sure who created the tables? Please create an "open_FRED" user group on the OEP and add all contributor. Then the table owner needs to set the group to the table permissions. By that we can update the metadata and more easy next time.

You can create the goup (add new member) and add permissions to a table using the OEP website (oep account required).

@christian-rli

Ludee commented 3 years ago

Thanks and rights are granted.

Ludee commented 3 years ago

The table with the variables is missing the units. That's an important information. @gnn

https://openenergy-platform.org/dataedit/view/climate/openfred_variables

OpenEnergyPlatform / data-preprocessing

review/openfred_weatherdata #47

Note: