search

OpenEnergyPlatform / data-preprocessing

Repository for data formatting, import of data, data and metadata review, and data curation.
GNU Affero General Public License v3.0
10 stars 7 forks source link

review/wind_turbine_library #26

Open christian-rli opened 5 years ago

christian-rli commented 5 years ago

The table has been renamed to: wind_turbine_library https://openenergy-platform.org/dataedit/view/supply/wind_turbine_library

The turbine library is a collection of wind turbine data. It is available here: https://openenergy-platform.org/dataedit/view/supply/wind_turbine_library ready to be reviewed and extended. The corresponding metadata string can be observed here.

For review guidelines you can have a look here.

christian-rli commented 5 years ago

During my upload of the metadata string i stumbled across the other files within that folder. They are all sql-files, which made me think. The idea in all of these is to first define a table at the final location with all column definitions, granting user privileges and adding the metadata, basically as a comment in the form of a json-string. It even includes scenario log entries. This is a great way to reconstruct the creation of the data in the database. However I would argue that this is something to be considered internal to the database. There is a very limited number of people with sql-access to the database. Access to it should all be going via oedialect, which then takes care of logging internally. Column descriptions should be sufficiently supplied within the metadata. For these reasons I only uploaded the metadata as a pure json-file. For completeness sake I should maybe also include the python upload script that uses oedialect? Feel free to convince me that another approach makes more sense.

Ludee commented 5 years ago

You are totally right! The "SQL-way" is limited to OEP developers. The API and oedicalect are not able to work with the metadata yet. This is very import. Is this something OMI will fix!?

christian-rli commented 5 years ago

OMI will support the translation of metadata json from older to newer versions and to rdf. Since there are a lot of old strings in the database, this is a prerequisite for a general oedialect support of metadata strings. So with OMI support is on its way, as of yet I'm just not how long it'll take.

jh-RLI commented 5 years ago

@christian-rli I updated the metadatastring for this table to oep-v1.4 and pushed the file here.

I also added the missing metadatastring as comment on table. I tried to use the updated version but it seems like the oep data view having trouble to display the string correctly. Because of that I added the old oep-v1.3 string.

@Ludee should the OEP data view work with the new metadata version?

christian-rli commented 5 years ago

Thank you @jh-RLI ! The new standard "OEP-1.4.0" is not implemented in OEP as of yet. @MGlauer is working on it however. So it should come soon. For now it's fine as version "OEP-1.3". ☮

euronion commented 5 years ago

Hi there! (thanks @coroa for pointing me here) We recently connected the turbine_library to our project Atlite and while writing the connector I came across some things which I would like to comment on (partially metadata, partially homogenisation in the data formats).

sorry if this is OT - maybe I missed the appropriate place for these questions: I am not sure how to contribute: Is there a how-to? (The review guidelines didn't answer this question for me). Also: Is the version in this issue already reflected in the OEDB. Because it changed since I checked last week and I failed to find any indicator, which version the database viewer is offering and when it was last updated?!

Hope you can get me up to speed so I can provide some additions. :)

Ludee commented 5 years ago

Hi @euronion,

this is the correct issue for the moment. We will set a link to the windpowerlib as well. We are happy to collaborate and are always open for comments and contributions!

We (@christian-rli) were working on the "official release" of the wind turbine library last week. The current version was for testing only. We discussed a renaming and fixed some power curves. The OEP metadata version has been updated as well. The data versioning of the OEP is not implemented in total yet. Working on it ;)

Ludee commented 5 years ago

After an internal group discussion we decided to rename the table and specify that we are collecting wind turbine specifications. The metadata is using the new version 1.4.0 and is not yet displayed on the OEP. The contributions can be made in the new ticket #28

christian-rli commented 5 years ago

@MGlauer uploading metadata in version 1.4.0 to the OEDB won't result in a correct display thereof. omi's v1.4 support was merged in this PR. What will it take to have it working on the OEP? Can I help somehow?

jh-RLI commented 3 years ago

updated the metadata as there was one field missing in resources: https://github.com/OpenEnergyPlatform/data-preprocessing/blob/master/data-curation/supply.wind_turbine_library_oep-1.4.0.json

SMEISEN commented 9 months ago

The table has been renamed to: wind_turbine_library https://openenergy-platform.org/dataedit/view/supply/wind_turbine_library

The turbine library is a collection of wind turbine data. It is available here: https://openenergy-platform.org/dataedit/view/supply/wind_turbine_library ready to be reviewed and extended. The corresponding metadata string can be observed here.

For review guidelines you can have a look here.

Hello, I would like to extend the database. Unfortunately, I could not find a description how to propose new entries.

jh-RLI commented 9 months ago

Great to hear!

It would be great if you could provide a file.csv here to show what you want to update. If the file contains a lot of entries, it's also fine to provide a subset of it.

I need to double check what our current process looks like. As I recall, we would need to move the table to the "model_draft" topic in the OEP, then upload the data, then move it back. This requires access permissions. I can assist you with this.