Open Ludee opened 6 years ago
Recent problems:
Rename SP-folder from TIMES PanEU to TIMESPanEU
Adjusted file name (missing underscore after date) 2017-11-15-StorageInnov_TIMESPanEU_FrameworkV1_DataV1_Output.xlsx 2017-11-15_StorageInnov_TIMESPanEU_FrameworkV1_DataV1_Output.xlsx
table | pathway | version | count |
---|---|---|---|
Input | Base | DataV1 | 77392 |
Input | Base | DataV2 | 81424 |
Input | HighRES | DataV1 | 78176 |
Input | Pilot | DataV1 | 23416 |
Input | StorageInnov | DataV1 | 77392 |
Output | Base | DataV1 | 420672 |
Output | Base | DataV2 | 437424 |
Output | Base(withRen.Target) | DataV1 | 421984 |
Output | HighRES | DataV1 | 421984 |
Output | Pilot | DataV1 | 421992 |
Output | Pilot2 | DataV1 | 421976 |
Output | StorageInnov | DataV1 | 421984 |
I found some inconsistencies in the numbering of the indicators. Every indicator should have a unique nid.
I think this is due to a bug in the database adapter and appears on the Base pathway of the output.
Hi @pkork,
the sources info in the metadata of the input model data of TIMES PanEU is incomplete, please follow this instruction to provide the infos.
In the excel, i provided I see that they are complete. The sources are either IER research or WP2 roadmaps. Do you refer to the information in the column P actually or you are asking additional work to do? I'm sorry it is not clear for me:(
In the TIMES input data, there is a column named "sources" you are refering to. @4lm is refering to the "metadata string" for the entire table. The metadata uses JSON syntax (key-value). Have a look at this example.
The link directs to the corresponding lines 62-64 for the input table:
"sources": [ {"name": "", "description": "", "url": "", "license": "", "copyright": ""}, {"name": "", "description": "", "url": "", "license": "", "copyright": ""} ],
You need to fill out these 5 keys! Use "none" if na. Adding "IER research" as your source is possible, but IMHO this is scientific BS. But I know it quite difficult to track every value and source.
Hi @pkork, hi @Ludee,
I wanted to get an understanding of all sources for the input data, so that I can assist adding them to the metadata JSON string. I had a look at the REEEM database and in the table "reeem_times_paneu_input" I aggregated all the sources for all pathways, frameworks and data versions using this SQL query:
SELECT source, COUNT(*) AS amount, ROUND((COUNT(*) * 100.0 / (SELECT COUNT(*) FROM model_draft.reeem_times_paneu_input)), 2) AS percentage
FROM model_draft.reeem_times_paneu_input
GROUP BY source;
As a result I get 2 sources:
source | amount | percentage |
---|---|---|
NULL | 32586 | 2.30 |
WP2 Roadmap F1_TI1_P1 | 30240 | 2.13 |
IER research | 1354142 | 95.57 |
Please note: The NULL values are from the Pilot pathway and the EU28 aggregations.
@pkork, for this two sources we now need a closer description, source-url, license and copyright terms. Please follow the general instruction to do so: https://github.com/ReeemProject/reeem_db/issues/55#issuecomment-492234763
Hi @pkork, any news concerning the source infos for the metadata?
Hi @pkork, also the license info in the metadata for the input data is stil set on tba: https://github.com/ReeemProject/reeem_db/blob/master/database_setup/reeem_db_setup_times_paneu.sql#L65-L71
Hi @pkork,
me again :)
There is a file in the Sharepoint Model_Data folder, named:
2019-05-06_All Pathways_TIMESPanEU_Input.xlsx
Two things:
Are you OK with this changes?
@Ludee, what do you think? Is this viable (especially my second "DataV0" suggestion)?
Edit: I also would set the missing framework value to "FrameworkV0"
EditEdit: @Ludee, I just see, that we used in another model "PathwayNA", so I assume "NA" stands for not available, we also could use this here. So, "DataNA" and "FrameworkNA" instead of "DataV0" and "FrameworkV0" - what do you think? I find both solutions OK, but we already used "NA" in such a case, so I would stick with this solution ...
I’m ok with the changes and I will come back to other points within next week. Regards, Pinar From: Alexis Michaltsis [mailto:notifications@github.com] Sent: Wednesday, May 29, 2019 5:04 PM To: ReeemProject/reeem_db reeem_db@noreply.github.com Cc: Pinar Korkmaz Pinar.Korkmaz@ier.uni-stuttgart.de; Mention mention@noreply.github.com Subject: Re: [ReeemProject/reeem_db] Database Adapter - TIMES PanEU (#4)
Hi @pkorkhttps://github.com/pkork,
me again :)
There is a file in the Sharepoint Model_Data folder, named:
2019-05-06_All Pathways_TIMESPanEU_Input.xlsx
Two things:
Are you OK with this changes?
@Ludeehttps://github.com/Ludee, what do you think? Is this viable (especially my second "DataV0" suggestion)?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ReeemProject/reeem_db/issues/4?email_source=notifications&email_token=AIBB3WM6FY72VYRYJSUNYYDPX2LOZA5CNFSM4EB6INL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWPUDNQ#issuecomment-496976310, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIBB3WKK3CECCXIQ6DCK4WLPX2LOZANCNFSM4EB6INLQ.
IMPORTANT: Request for OEP-Publishing
Hi @JuliaIER and @pkork,
you licensed reeem_times_paneu_output as CC-BY-4.0 and therefore suitable for publishing on the Open Energy Platform (OEP). We now pre-staged the data on OEP and are ready to hit the button :) Nevertheless we want to ask for your final OK!
If you want to have a look at the data, you find it in the REEEM DB in model_draft.reeem_times_paneu_output
Thanks in advance!
Hi @JuliaIER and @pkork,
a little reminder, we still need the input sources and license info of TIMES for the input metadata.
Please provide us now with the metadata of your data, just post it in a comment in this issue thread!
SOURCES: "REEEM Times PanEU Input"
"sources": [
{"name": "none", "description": "none", "url": "none", "license": "none", "copyright": "none"},
{"name": "none", "description": "none", "url": "none", "license": "none", "copyright": "none"} ],
LICENSE: "REEEM Times PanEU Input"
"license":
{"id": "none",
"name": "none",
"version": "none",
"url": "none",
"instruction": "none",
"copyright": "none"},
Thanks!
CC: @Ludee
IMPORTANT: Request for OEP-Publishing
Hi @JuliaIER and @pkork,
you licensed reeem_times_paneu_output as CC-BY-4.0 and therefore suitable for publishing on the Open Energy Platform (OEP). We now pre-staged the data on OEP and are ready to hit the button :) Nevertheless we want to ask for your final OK!
If you want to have a look at the data, you find it in the REEEM DB in model_draft.reeem_times_paneu_output
Thanks in advance!
Hi @JuliaIER and @pkork, what about this request? What is the status here?
CC: @Ludee
modeller: @pkork