Ludee commented 6 years ago

modeller: Hauke

Ludee commented 6 years ago

Please change document structure regarding #5!

[x] Add columns schema and field
[x] Rename Table to category
[x] For all Tables (Input / Output)

See SP Test_data Input file sheet "AT" for additional hints and comments!

4lm commented 5 years ago

Hi Hauke,

we have input and output data of the OSeMBE model in REEEM DB, but we are missing some metadata describing the data ... please provide us now with the metadata of your data, just post it in a comment in this issue thread. This is the two metadata strings we have so far, everywhere you read "none", we need infos (except in the "contributors" and "fields" sections) - most important are sources and under which licenses you distribute your output data.

INPUT:

{
    "title": "REEEM OSeMBE Input",
    "description": "Input parameters: Service Demands, Fuel Prices, Potentials",
    "language": [ "eng" ],
    "spatial": 
        {"location": "none",
        "extent": "Europe",
        "resolution": "Country"},
    "temporal": 
        {"reference_date": "2015",
        "start": "2015",
        "end": "2060",
        "resolution": "1 year"},
    "sources": [
        {"name": "", "description": "", "url": "", "license": "", "copyright": ""},
        {"name": "", "description": "", "url": "", "license": "", "copyright": ""} ],
    "license": 
        {"id": "tba",
        "name": "tba",
        "version": "tba",
        "url": "tba",
        "instruction": "tba",
        "copyright": "tba"},
    "contributors": [
        {"name": "Ludee", "email": "none", "date": "2017-07-24", "comment": "Create table"},
        {"name": "Ludee", "email": "none", "date": "2017-11-14", "comment": "Update structure and metadata"},
        {"name": "Ludee", "email": "none", "date": "2018-01-30", "comment": "New naming structure"} ],
    "resources": [
        {"name": "model_draft.reeem_osembe_input",
        "format": "PostgreSQL",
        "fields": [
            {"name": "id", "description": "Unique identifier", "unit": "none"},
            {"name": "nid", "description": "Row id", "unit": "none"},
            {"name": "dfid", "description": "Internal dataframe id", "unit": "none"},
            {"name": "pathway", "description": "REEEM pathway", "unit": "none"},
            {"name": "framework", "description": "REEEM framework", "unit": "none"},
            {"name": "version", "description": "REEEM version", "unit": "none"},
            {"name": "schema", "description": "1. classification", "unit": "none"},
            {"name": "category", "description": "2. classification", "unit": "none"},
            {"name": "tags", "description": "Free classification", "unit": "none"},
            {"name": "field", "description": "Old classification", "unit": "none"},
            {"name": "region", "description": "Country or region", "unit": "none"},
            {"name": "year", "description": "Year", "unit": "none"},
            {"name": "indicator", "description": "Parameter name", "unit": "none"},
            {"name": "value", "description": "Parameter value", "unit": "unit"},
            {"name": "unit", "description": "Parameter unit", "unit": "none"},
            {"name": "aggregation", "description": "True if aggregated", "unit": "none"},
            {"name": "updated", "description": "Timestamp", "unit": "none"},
            {"name": "source", "description": "Data source", "unit": "none"} ] } ],
    "metadata_version": "1.3"}'

OUTPUT:

{
    "title": "REEEM OSeMBE Output",
    "description": "Output parameters: ",
    "language": [ "eng" ],
    "spatial": 
        {"location": "none",
        "extent": "Europe (EU28+NO+CH)",
        "resolution": "Country"},
    "temporal": 
        {"reference_date": "2015",
        "start": "2015",
        "end": "2050",
        "resolution": "1 year"},
    "sources": [
        {"name": "OSeMOSYS OSEMBE", "description": "The Open-Source energy Model Base for the European Union", "url": "http://www.osemosys.org/osembe.html", "license": "Apache License 2.0 (Apache-2.0)", "copyright": "tba"},
        {"name": "OSeMBE – An open-source engagement model", "description": "Model documentation", "url": "http://www.osemosys.org/uploads/1/8/5/0/18504136/osembe_documentation.pdf", "license": "none", "copyright": "none"},
        {"name": "OSeMBE input data", "description": "Data", "url": "http://www.osemosys.org/uploads/1/8/5/0/18504136/osembe_v1_20180821.zip", "license": "Open Data Commons Attribution License 1.0 (ODC-BY-1.0)", "copyright": "© Hauke Henke"}],
    "license": 
        {"id": "tba",
        "name": "tba",
        "version": "tba",
        "url": "tba",
        "instruction": "tba",
        "copyright": "tba"},
    "contributors": [
        {"name": "Ludee", "email": "none", "date": "2017-07-24", "comment": "Create table"},
        {"name": "Ludee", "email": "none", "date": "2017-11-14", "comment": "Update structure and metadata"},
        {"name": "Ludee", "email": "none", "date": "2018-01-30", "comment": "New naming structure"},
        {"name": "Ludee", "email": "none", "date": "2018-10-09", "comment": "Update table name and metadata"} ],
    "resources": [
        {"name": "model_draft.reeem_osembe_output",
        "format": "PostgreSQL",
        "fields": [
            {"name": "id", "description": "Unique identifier", "unit": "none"},
            {"name": "nid", "description": "Row id", "unit": "none"},
            {"name": "dfid", "description": "Internal dataframe id", "unit": "none"},
            {"name": "pathway", "description": "REEEM pathway", "unit": "none"},
            {"name": "framework", "description": "REEEM framework", "unit": "none"},
            {"name": "version", "description": "REEEM version", "unit": "none"},
            {"name": "schema", "description": "1. classification", "unit": "none"},
            {"name": "category", "description": "2. classification", "unit": "none"},
            {"name": "tags", "description": "Free classification", "unit": "none"},
            {"name": "field", "description": "Old classification", "unit": "none"},
            {"name": "region", "description": "Country or region", "unit": "none"},
            {"name": "year", "description": "Year", "unit": "none"},
            {"name": "indicator", "description": "Parameter name", "unit": "none"},
            {"name": "value", "description": "Parameter value", "unit": "unit"},
            {"name": "unit", "description": "Parameter unit", "unit": "none"},
            {"name": "aggregation", "description": "True if aggregated", "unit": "none"},
            {"name": "updated", "description": "Timestamp", "unit": "none"},
            {"name": "source", "description": "Data source", "unit": "none"} ] } ],
    "metadata_version": "1.3"}

HauHe commented 5 years ago

Hi @4lm, I have now updated the license and source section. Please see below. Anyhow, before publishing, @Ludee is the CC-BY-4.0 a suitable license for the output data? And @Ludee and @4lm for the out put data under resources it says ""name": "model_draft.reeem_osembe_output"" what does this refer to?

INPUT:

{ "title": "REEEM OSeMBE Input", "description": "Input parameters: Service Demands, Fuel Prices, Potentials", "language": [ "eng" ], "spatial": {"location": "none", "extent": "Europe", "resolution": "Country"}, "temporal": {"reference_date": "2015", "start": "2015", "end": "2060", "resolution": "15 timeslices per year"}, "sources": [ {"name": "Energy Technology Reference Indicator (ETRI) projections for 2010-2050", "description": "The Strategic Energy Technology Plan (SET-Plan) is the technology pillar of the EU's energy and climate policy. This report contains assessments of energy technology reference indicators (ETRI) and it is aimed at providing independent and up-to-date cost and performance characteristics of the present and future European energy technology portfolio.", "url": "https://publications.europa.eu/en/publication-detail/-/publication/79a2ddbd-5ba1-4380-93af-2ce274a840f0/language-en", "license": "unknown", "copyright": "European Union"}, {"name": "Catalog of CHP Technologies", "description": "The five technologies described in the Guide make up 97 percent of the CHP projects in place today and 99 percent of the total installed CHP electric capacity.", "url": "https://www.energy.gov/sites/prod/files/2016/09/f33/catalog_of_chp_technologies.pdf", "license": "unknown", "copyright": "unknown"}, {"name": "WEIO 2014 PG Assumptions", "description": "Power Generation in the New Policies and 450 Scenarios, Assumed investment costs, operation and maintenance costs and efficiencies in the IEA World Energy Investment Outlook 2014", "url": "no longer available", "license": "unknown", "copyright": "OECD/IEA"}, {"name": "EU Reference Scenario 2016", "description": "This report is an update of the previous Reference Scenario published in 2013. It focuses on the EU energy system, transport and greenhouse gas (GHG) emission developments, including specific sections on emission trends not related to energy, and on the various interactions among policies in these sectors.", "url": "https://publications.europa.eu/en/publication-detail/-/publication/aed45f8e-63e3-47fb-9440-a0a14370f243/language-en", "license": "unknown", "copyright": "European Union"}, {"name": "IEA ETSAP - Technology Brief E03", "description": "Technology brief on Nuclear power", "url": "http://109.73.233.125/~ieaetsap/E-TechDS/PDF/E03-Nuclear-Power-GS-AD-gct.pdf", "license": "unknown", "copyright": "IEA ETSAP"}, {"name": "Renewable Energy prospects: Germany, REmap 2030 analysis", "description": "IRENA REmap report identifying the gap between national renewable targets and the national renewable potential.", "url": "http://www.irena.org/publications/2015/Nov/Renewable-Energy-Prospects-Germany", "license": "unknown", "copyright": "IRENA"}, {"name": "REmap 2030 Renewable Energy Prospects for Poland", "description": "IRENA REmap report identifying the gap between national renewable targets and the national renewable potential.", "url": "http://www.irena.org/publications/2015/Oct/REmap-2030-Renewable-Energy-Prospects-for-Poland", "license": "unknown", "copyright": "IRENA"}, {"name": "Platts World Electric Power Plants Database", "description": "Data base of power plants with unit aggregation.", "url": "https://www.platts.com/products/world-electric-power-plants-database", "license": "unknown", "copyright": "S&P Global Platts"} , {"name": "Wind in power", "description": "Annual report by the European Wind Energy Association, now WindEurope, on wind power statistics and installed wind power capacities.", "url": "https://windeurope.org/about-wind/statistics/european/wind-in-power-2015/", "license": "unknown", "copyright": "WindEurope asbl/vzw"}, {"name": "Renewable Energy Statistics 2017", "description": "Statistics on renewable production, capacities and related financial flows.", "url": "http://www.irena.org/DocumentDownloads/Publications/IRENA_Renewable_Energy_Statistics_2017.pdf", "license": "unknown", "copyright": "IRENA"}, {"name": "Renewable Power Generation Costs in 2014", "description": "Report on the the cost of renewable power generation, used for deriving biomass costs", "url": "http://www.irena.org/publications/2015/Jan/Renewable-Power-Generation-Costs-in-2014", "license": "unknown", "copyright": "IRENA"}, {"name": "Production of Liquid Biofuels", "description": "Technology brief on the production of biofuels and its costs.", "url": "https://iea-etsap.org/E-TechDS/PDF/P10IR_Liq%20Biof_GS_Jan2013_final_GSOK.pdf", "license": "unknown", "copyright": "IEA-ETSAP and IRENA"}, {"name": "DECC 2015 Fossil Fuel Price Assumptions", "description": "The report updates DECC’s long-term price projections for oil, gas and coal.", "url": "https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/477958/2015_DECC_fossil_fuel_price_assumptions.pdf", "license": "Open Government License", "copyright": "Crown copyright"}, {"name": "The Economics of Nuclear Power", "description": "Webpage providing information on the operating cost of nuclear PP including the fuel cost.", "url": "http://www.world-nuclear.org/information-library/economic-aspects/economics-of-nuclear-power.aspx", "license": "unknown", "copyright": "World Nuclear Association"}, {"name": "ENTSO-E Transparency Platform", "description": "Central collection and publication of electricity generation, transportation and consumption data and information for the pan-European market.", "url": "https://transparency.entsoe.eu/dashboard/show", "license": "unknown", "copyright": "unknown"}, {"name": "Renewables-ninja", "description": "Renewables.ninja allows you to run simulations of the hourly power output from wind and solar power plants located anywhere in the world.", "url": "www.renewables.ninja", "license": "CC BY-NC 4.0", "copyright": "unknown"}, {"name": "EU Emissions Trading System (ETS) data viewer", "description": "The EU ETS data viewer provides an easy access to emission trading data contained in the European Union Transaction Log (EUTL).", "url": "https://www.eea.europa.eu/data-and-maps/dashboards/emissions-trading-viewer-1", "license": "CC BY 2.5 DK", "copyright": "unknown"}, {"name": "Total gross electricity generation", "description": "Data on the total gross electricity generation in the EU.", "url": "http://ec.europa.eu/eurostat/tgm/table.do?tab=table&init=1&plugin=1&language=en&pcode=ten00087", "license": "unknown", "copyright": "European Union"}, {"name": "CO2 emission intensity", "description": "CO2 emission intensity of electricity in the EU by year.", "url": "https://www.eea.europa.eu/data-and-maps/daviz/co2-emission-intensity-3", "license": "CC BY 2.5 DK", "copyright": "European Environment Agency (EEA)"}, {"name": "Forecast of energy demand in Norway towards 2050", "description": "The projection of energy demand is based on current policies, and is made for all energy carriers, and includes both stationary energy demand and transport for the analysis period 2010-2050.", "url": "https://www.ntnu.edu/documents/1261967833/1262617931/Energy+demand+in+Norway+towards+2050.pdf/80f05210-7380-4c19-b9ed-52801b9d3fe1", "license": "unknown", "copyright": "unknown"}, {"name": "CenSES Energy demand projections towards 2050 - Reference path", "description": "A Position Paper prepared by FME CenSES", "url": "https://www.ntnu.edu/documents/1261967833/1262617931/Position-paper_Energy-Projections_utenbleed.pdf/0949b222-5d1b-4b06-9217-9287cb578b6a", "license": "unknown", "copyright": "unknown"}, {"name": "Electricity statistics", "description": "The electricity statistics evaluate the production of electricity and the consumption of electrical energy in Switzerland.", "url": "http://www.bfe.admin.ch/themen/00526/00541/00542/00630/index.html?lang=en&dossier_id=04840", "license": "unknown", "copyright": "Swiss federal authorities"}, {"name": "Geothermal energy Switzerland", "description": "Collection of information geothermal energy in general and in specific for Switzerland.", "url": "http://www.bfe.admin.ch/themen/00490/00501/index.html?lang=en", "license": "unknown", "copyright": "Swiss federal authorities"}, {"name": "Potenzial - Geothermie Schweiz", "description": "Estimation of the potential of geothermal-energy in Switzerland useable for electricity production", "url": "https://geothermie-schweiz.ch/waermestrom/petrothermal/potenzial/", "license": "unknown", "copyright": "Geothermie-Schweiz.ch"}, {"name": "Wasserkraftpotenzial der Schweiz", "description": "Report on current situation and potential of hydro power in Switzerland", "url": "https://www.swv.ch/wp-content/uploads/2018/03/Faktenblatt-Wasserkraftpotenzial_SWV.pdf", "license": "unknown", "copyright": "Schweizerischer Wasserwirtschaftsverband"}, {"name": "OECD iLibrary - Waste", "description": "This database provides the latest and most comprehensive statistics across OECD countries on waste.", "url": "https://www.oecd-ilibrary.org/environment/data/oecd-environment-statistics/municipal-waste_data-00601-en", "license": "unknown", "copyright": "OECD"}, {"name": "Facts 2015 Energy and water resources in Norway", "description": "Report on energy and water resources in Norway", "url": "https://www.regjeringen.no/en/aktuelt/fakta-2015-energi--og-vannressurser-i-norge/id2395568/", "license": "unknown", "copyright": "unknown"}, {"name": "Wind Energy in Estonia", "description": "Status of Wind power in Estonia reported on the webpage of the Estonian Wind power Association", "url": "http://www.tuuleenergia.ee/en/windpower-101/statistics-of-estonia/", "license": "unknown", "copyright": "unknown"}, {"name": "Wind energy statistics in Finland", "description": "Report on installed wind power capacities and power generation by wind power.", "url": "https://www.vttresearch.com/services/low-carbon-energy/wind-energy/wind-energy-statistics-in-finland", "license": "unknown", "copyright": "VTT"}, {"name": "Wind energy assessment considering geographic and environmental restrictions in Sweden: A GIS-based approach", "description": "Journal paper on wind power potential in Sweden.", "url": "https://www.sciencedirect.com/science/article/pii/S0360544215001991?via%3Dihub", "license": "unknown", "copyright": "Elsevier Ltd."}, {"name": "Gesetz über die friedliche Verwendung der Kernenergie und den Schutz gegen ihre Gefahren (Atomgesetz)", "description": "Atomgesetz in der Fassung der Bekanntmachung vom 15. Juli 1985 (BGBI. I S. 1565), das zuletzt durch Artikel 1 des Gesetzes vom 10. Juli 2018 (BGBI. I S. 1122) geändert worden ist", "url": "https://www.gesetze-im-internet.de/atg/AtG.pdf", "license": "unknown", "copyright": "unknown"}, {"name": "Potenziale, Kosten und Umweltauswirkungen von Stromproduktionsanlagen - Synthese", "description": "Report on the potentials of different power generation technologies in Switzerland.", "url": "https://www.newsd.admin.ch/newsd/message/attachments/50263.pdf", "license": "unknown", "copyright": "unknown"}, {"name": "Nuclear energy", "description": "Description of history and current situation of nuclear energy for electricity generation in Switzerland.", "url": "http://www.bfe.admin.ch/themen/00511/index.html?lang=en", "license": "unknown", "copyright": "Swiss federal authorities"}, {"name": "National generation capacity", "description": "Open Source power system data, installed power generation capacities in Europe", "url": "https://data.open-power-system-data.org/national_generation_capacity/2019-02-22", "license": "unknown", "copyright": "unknown"}, {"name": "FE08: Average Cost of Fuels and Energy Consumed by Enterprises by Type of Fuel/Energy", "description": "Fuel cost of oil shale in Estonia", "url": "http://pub.stat.ee/px-web.2001/Dialog/varval.asp?ma=FE08&lang=1", "license": "unknown", "copyright": "unknown"}, {"name": "Statistics World - Total Primary Energy Supply (TPES) by source (table)", "description": "Electricity generation by source an country", "url": "https://www.iea.org/statistics/?country=WORLD&year=2016&category=Energy%20supply&indicator=TPESbySource&mode=table&dataTable=ELECTRICITYANDHEAT", "license": "non-exclusive", "copyright": "OECD/IEA"}, {"name": "Nuclear Power in Poland", "description": "A history, status and look out on nuclear power in Poland.", "url": "https://www.world-nuclear.org/information-library/country-profiles/countries-o-s/poland.aspx", "license": "unknown", "copyright": "2016-2020 World Nuclear Association"}, {"name": "Nuclear Power in Slovenia", "description": "A history, status and look out on nuclear power in Slovenia.", "url": "https://www.world-nuclear.org/information-library/country-profiles/countries-o-s/slovenia.aspx", "license": "unknown", "copyright": "2016-2020 World Nuclear Association"} ], "license": {"id": "CC-BY-4.0", "name": "Creative Commons Attribution 4.0 International", "version": "4.0", "url": "https://spdx.org/licenses/CC-BY-4.0.html", "instruction": "You are free: To Share, To Create, To Adapt; As long as you: Attribute!", "copyright": "© Hauke Henke"}, "contributors": [ {"name": "Ludee", "email": "none", "date": "2017-07-24", "comment": "Create table"}, {"name": "Ludee", "email": "none", "date": "2017-11-14", "comment": "Update structure and metadata"}, {"name": "Ludee", "email": "none", "date": "2018-01-30", "comment": "New naming structure"} ], "resources": [ {"name": "model_draft.reeem_osembe_input", "format": "PostgreSQL", "fields": [ {"name": "id", "description": "Unique identifier", "unit": "none"}, {"name": "nid", "description": "Row id", "unit": "none"}, {"name": "dfid", "description": "Internal dataframe id", "unit": "none"}, {"name": "pathway", "description": "REEEM pathway", "unit": "none"}, {"name": "framework", "description": "REEEM framework", "unit": "none"}, {"name": "version", "description": "REEEM version", "unit": "none"}, {"name": "schema", "description": "1. classification", "unit": "none"}, {"name": "category", "description": "2. classification", "unit": "none"}, {"name": "tags", "description": "Free classification", "unit": "none"}, {"name": "field", "description": "Old classification", "unit": "none"}, {"name": "region", "description": "Country or region", "unit": "none"}, {"name": "year", "description": "Year", "unit": "none"}, {"name": "indicator", "description": "Parameter name", "unit": "none"}, {"name": "value", "description": "Parameter value", "unit": "unit"}, {"name": "unit", "description": "Parameter unit", "unit": "none"}, {"name": "aggregation", "description": "True if aggregated", "unit": "none"}, {"name": "updated", "description": "Timestamp", "unit": "none"}, {"name": "source", "description": "Data source", "unit": "none"} ] } ], "metadata_version": "1.3"}'

OUTPUT

{ "title": "REEEM OSeMBE Output", "description": "Output parameters: ", "language": [ "eng" ], "spatial": {"location": "none", "extent": "Europe (EU28+NO+CH)", "resolution": "Country"}, "temporal": {"reference_date": "2015", "start": "2015", "end": "2050", "resolution": "15 timeslices per year"}, "sources": [ {"name": "OSeMOSYS OSEMBE", "description": "The Open-Source energy Model Base for the European Union", "url": "http://www.osemosys.org/osembe.html", "license": "Apache License 2.0 (Apache-2.0)", "copyright": "tba"}, {"name": "OSeMBE – An open-source engagement model", "description": "Model documentation", "url": "http://www.osemosys.org/uploads/1/8/5/0/18504136/osembe_documentation.pdf", "license": "none", "copyright": "none"}, {"name": "OSeMBE input data", "description": "Data", "url": "http://www.osemosys.org/uploads/1/8/5/0/18504136/osembe_v1_20180821.zip", "license": "Creative Commons Attribution 4.0 International (CC-BY-4.0)", "copyright": "© Hauke Henke"}], "license": {"id": "CC-BY-4.0", "name": "Creative Commons Attribution 4.0 International", "version": "4.0", "url": "https://spdx.org/licenses/CC-BY-4.0.html", "instruction": "You are free: To Share, To Create, To Adapt; As long as you: Attribute!", "copyright": "© Hauke Henke"}, "contributors": [ {"name": "Ludee", "email": "none", "date": "2017-07-24", "comment": "Create table"}, {"name": "Ludee", "email": "none", "date": "2017-11-14", "comment": "Update structure and metadata"}, {"name": "Ludee", "email": "none", "date": "2018-01-30", "comment": "New naming structure"}, {"name": "Ludee", "email": "none", "date": "2018-10-09", "comment": "Update table name and metadata"} ], "resources": [ {"name": "model_draft.reeem_osembe_output", "format": "PostgreSQL", "fields": [ {"name": "id", "description": "Unique identifier", "unit": "none"}, {"name": "nid", "description": "Row id", "unit": "none"}, {"name": "dfid", "description": "Internal dataframe id", "unit": "none"}, {"name": "pathway", "description": "REEEM pathway", "unit": "none"}, {"name": "framework", "description": "REEEM framework", "unit": "none"}, {"name": "version", "description": "REEEM version", "unit": "none"}, {"name": "schema", "description": "1. classification", "unit": "none"}, {"name": "category", "description": "2. classification", "unit": "none"}, {"name": "tags", "description": "Free classification", "unit": "none"}, {"name": "field", "description": "Old classification", "unit": "none"}, {"name": "region", "description": "Country or region", "unit": "none"}, {"name": "year", "description": "Year", "unit": "none"}, {"name": "indicator", "description": "Parameter name", "unit": "none"}, {"name": "value", "description": "Parameter value", "unit": "unit"}, {"name": "unit", "description": "Parameter unit", "unit": "none"}, {"name": "aggregation", "description": "True if aggregated", "unit": "none"}, {"name": "updated", "description": "Timestamp", "unit": "none"}, {"name": "source", "description": "Data source", "unit": "none"} ] } ], "metadata_version": "1.3"}

4lm commented 5 years ago

Hi @HauHe,

I have now updated the license and source section. Please see below.

Thanks. I will update the metadata.

Anyhow, before publishing, @Ludee is the CC-BY-4.0 a suitable license for the output data?

OK. I will wait with the transfer of the data to the OEP until you talked with @Ludee.

And @Ludee and @4lm for the out put data under resources it says ""name": "model_draft.reeem_osembe_output"" what does this refer to?

This refers to the database schema and table, and also the database format.

    "resources": [
        {"name": "model_draft.reeem_osembe_output",
        "format": "PostgreSQL",
        ...

4lm commented 5 years ago

Hi @HauHe,

@Ludee told me that we have your go for the publication of the OSeMBe model input and output data. I initiated the final step an copied the data in the REEEM database tables reeem_osembe_input and reeem_osembe_output to and internal schema in the OEP database. So we are good to go for the final publication on OEP :)

If you want to have a look at the data, you find it in the REEEM DB in

model_draft.reeem_osembe_input model_draft.reeem_osembe_output

Regards!

HauHe commented 5 years ago

Hi @4lm , thanks for the heads up. I actually just contacted our administration to check the situation concerning the IP. I.e. I'm making sure if I'm the copyright holder or KTH as my employer. Could you perhaps delay the publication till I've sorted that? Best regards!

HauHe commented 5 years ago

Hi @Ludee , you mentioned in Brussels that you have a script for getting the excel file names in the directory from where you want to upload to the database. Is it here on GitHub? How is it called or could you please share? @4lm I guess you know of that script as well. Would be great if you could share it. Best regards!

4lm commented 5 years ago

Could you perhaps delay the publication till I've sorted that?

OK, we are waiting! Ping: @Ludee

that you have a script for getting the excel file names in the directory ... Would be great if you could share it.

What do you want to do with it? Its better if we upload the data to the REEEM DB, to keep track and avoid technical issues like double entries ... but, if you just want to have a look, here is the link to the OSeMBE adapter script I think you talking about: https://github.com/ReeemProject/reeem_db/blob/master/database_adapter/reeem_adapter_osembe.py

HauHe commented 5 years ago

Hi @4lm , hi @Ludee , I checked with KTH and with the REEEM grant agreement and the IP seems to be with me. So please publish under my name.

I had thought of doing the uploads. But I'm happy of you want to do that. I have now started putting the new data on sharepoint. At the moment some input data. Output will follow today and tomorrow. Would be great if you could start the upload.

Cheers

4lm commented 5 years ago

I had thought of doing the uploads. But I'm happy of you want to do that. I have now started putting the new data on sharepoint. At the moment some input data. Output will follow today and tomorrow. Would be great if you could start the upload.

Hi @HauHe,

are we complete now on Sharepoint? Are the files from yesterday the last ones or are more files coming?

I will start the import of the data into the REEEM DB, now! If there are more files to come, please put them in the separate sub-folder, so that I don't get confused :wink:

CC: @Ludee

4lm commented 5 years ago

Hi @HauHe, import is running. 187 files. Taking the experience of the last batch import into account, this will take 10+ days. I will give notice, when it's done!

HauHe commented 5 years ago

Hi @4lm , thanks for starting the upload! What is the status. I noticed that there are some file missing. Have added them now on sharepoint (52 more files). Could you please upload them as well? Thanks a lot for your support!

4lm commented 5 years ago

Hi @HauHe, 28 of 187 files to go (I had a closed connection to the database twice, so lost a few days). Around 15 files take one day to import. So your new 52 files could be taken into the queue on Thursday and will be done (if everything works out) by Monday or Tuesday.

4lm commented 5 years ago

Hi @HauHe,

the upload of the 28 files went through. I'm uploading now your last delivery (52 files) from Monday. This will take ~ 4 days.

From tomorrow on, I'm on holiday until the 9th of September.

4lm commented 5 years ago

Hi @HauHe,

the last 52 files went through. We are done :rocket:

HauHe commented 5 years ago

Hi @4lm , great that are good news! Thanks a lot!

ReeemProject / reeem_db

Database Adapter - OSeMBE #13

INPUT:

OUTPUT