Closed henhuy closed 1 year ago
Download of raw_data
could be implemented easily by using function download_table_from_OEP
from oemof_b3.tools.oep
Downloading raw data is much easier to implement into the pipeline than resources. Something like:
rule get_raw_data:
outputs: "raw/..."
scripts:
if config.raw.source == "oep"
download_from_oep
elif config.raw.source == "zenodo"
download_from_zenodo
elif config.raw.source == "manual"
logger.warning("Have to provide raw data manually by copying it to raw")
This is because there is no rule for raw data yet, but there are many rules for creating resources.
Idea: Implement a switch in Snakefile that allows to select which resources should be used, either results/_resources or results/_oep_resources.
Which rule uses the resources?
rule build_datapackage
takes resources (and some raw data) as input, as specified in the scenario_yaml. This makes it a bit more tricky.
Found a solution for the above problem: Moving all rules that prepare resources in a submodule that can be included or not depending on a value in the settings. #290
Almost there. Still missing:
The resources can be downloaded, but there are differences to the ones created locally. First, the delimiter is different. Second, entries are in "". Added to the TODO list in first post.
Added script
download_resources
which downloads resources from OEP and saves them in folderresults/_resources
. TODOS:download_resources.py
@jnnr could you complete TODOS above, as you might know better all resources and related tables on OEP. and you are more familiar with snakemake...?
,
, but are expected;
""