search

OpenEnergyPlatform / data-preprocessing

Repository for data formatting, import of data, data and metadata review, and data curation.
GNU Affero General Public License v3.0
10 stars 7 forks source link

Review: EMSIG #84

Closed lechlm closed 3 years ago

lechlm commented 3 years ago

Issue description

I'm submitting an open dataset with a corresponding OEMetadata string for review. Please see review_process.md for technical detail. The dataset shows recordings of a productive Energy Management System based on OpenEMS Edge. It contains a total of 3 tables, one containing the energy data recorded by the decentralized household energy management systems (EMS), one containing energy management systems with their corresponding storage system and the last one containing energy storage systems with their respective model, vendor, capacity, power and battery type.

Workflow checklist

  1. GitHub

    • [x] I have submitted this issue to have metadata and data review documented (Issue #NR)
    • [x] Create a new review-branch and push OEMetadata to new branch (review/project_nameofdata#NR). If this step is too difficult, attach a file with the metadata as a comment in this issue and let the reviewer know.

  2. OEP

    • [x] Upload data to the OEP in schema model_draft (see upload tutorial)
    • [x] Link URL of data in this issue (model_draft.project_nameofdata)

  3. Start a Review

    • [x] Start a pull request (PR) from review-branch to master
    • [x] Assign a reviewer and get in contact

  4. Reviewer section

    • [x] A reviewer starts working on the issue
    • [x] Review data license
    • [x] A reviewer finished working on this issue (and awarded a badge)
    • [x] Update metadata on table
    • [x] Data moved to its final schema
    • [x] Add OEP tags to table
    • [x] Merge PR and delete review-branch
    • [x] Document final links of metadata and data in issue description
    • [x] Close issue

Metadata and data for review

Here are the links to my data and metadata. Naming follows the pattern model_draft.project_nameofdata: Metadata:

86

Data:

https://openenergy-platform.org/dataedit/view/model_draft/emsig_energy_management_systems https://openenergy-platform.org/dataedit/view/model_draft/emsig_energy_storage_systems https://openenergy-platform.org/dataedit/view/model_draft/emsig_energy_data_by_ems

Reviewed and published metadata and data

final location of data:

final location of metadata:

jh-RLI commented 3 years ago

@lechlm sorry for the late response. I also received your email to @Ludee who is currently not available. I don't have the permissions to add you to the appropriate Github group so that you can create a PullRequest(PR) and push your information there. @christain-rli has the necessary permissions, but he is still on holiday until 31.05. We should be able to continue from next week. Alternatively, you can upload the metadata here in the issue as a .zip archive and I can create the PR for you.

I assume that you already know the next steps? Do you already have an OEP-website user account? Have you already created the metadata for your data? Currently, in addition to manually filling in a JSON string, there is also the option to use the OEP website. To do so, follow the link to the tables you have created and use the button "Edit" (see screenshot at 1. below.) You should also create/add a User Group with "write" permissions to your table so at least @christian-rli and I can edit your table to update the metadata and so on. (see 2. in the screenshot below). My OEP-Website username is "Jonas.Huber" I'm not sure what @christian-rli user name is ATM. image

lechlm commented 3 years ago

@jh-RLI Thank you for the advices! Today, I was able to create the PR (see email from Mr. Hülk). Metadata is also available on the OEP-website now, initially I thought providing it via the PR is enough. I further added a User Group, currently only with your user.

christian-rli commented 3 years ago

Thank you @jh-RLI . My OEP username is "Christian Hofmann". I didn't look further into the metadata, but at a first glance, it looks comprehensive. Also, ODbL is a suitable license ✓.

jh-RLI commented 3 years ago

@lechlm Glad I could help :). It would have been enough to create a PR and provide the metadata there as "raw" JSON files. I just wanted to point out the relatively new feature, as the metadata is somewhat easier to fill in/create via the OEP website.

jh-RLI commented 3 years ago

@lechlm I have now finished reviewing all the metadata from the PR. They turned out very well! I only adjusted a few small things (see PR). With the Gold Bage, the data can be published like this. The Platinum Bage can be awarded when several reviewers have had a look at it. The next step is to move the data into the appropriate target schema. Which of the thematic schemes seems most suitable to you? List of currently available schemes -> OEP-schemes

lechlm commented 3 years ago

@jh-RLI Yes, metadata creation via the website is definitely easier, thanks for the hint. And thank you for reviewing the metadata! The 'demand' schema seems most promising because the aggregated profiles from our FEMS kind of reflect the new demand of PV+Battery equipped households. Besides, I added @christian-rli to the User Group EMSIG.

christian-rli commented 3 years ago

Congratulations on a smooth review and thank you for adding me to the user group @lechlm . Moving tables into final schemas via the api is currently being reworked. I will move your tables to the 'demand' schema as soon as possible and let you know here right away, once it's done. Until then the tables will remain available in the 'model_draft' schema. Sorry for the wait.

lechlm commented 3 years ago

I understand, thank you very much, @christian-rli !

christian-rli commented 3 years ago

Sorry for the wait @lechlm . The implementation took longer than anticipated. I just moved the tables into the demand schema (where they can be filtered for with the tag EMSIG: https://openenergy-platform.org/dataedit/view/demand?query=EMSIG ).

I'll go through the last steps and merge the PR. Do feel free to comment again in the issue if there are any remaining questions.