Unable to download base data by using pypsa-earth subworkflow

Emre-Yorat89 commented 7 months ago

Checklist

[ ] I am using the current main branch or the latest release. Please indicate.
[ ] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f pypsa-earth/envs/environment.yaml.

Describe the Bug

Currently, pypsa-distribution main workflow is working. However, I could not run pypsa-earth subworkflow. For example; an error message is given for add_electricity rule. By running snakemake -j1 add_electricity, it tries to access natura.tiff file but the file is not downloaded at the first place by pypsa-earth subworkflow with retrieve_databundle_script. Thank you for your suggestions to solve this problem.

Error Message

Rule_addElectricity

<paste here>

ekatef commented 6 months ago

Hey @Emre-Yorat89, thanks a lot for reporting. I had the same issue which should be now fixed thanks to the recent changes in the main. Could you please try if it works for you, as well?

Emre-Yorat89 commented 6 months ago

Hey @ekatef , I will try tomorrow and inform you with the outcome, thank you for fixing the error. Apart from this error, I am also unable to run any pypsa-earth related rule such as generating renewable profiles with WSL. May be the other errors are corrected by your fixes too.

davide-f commented 6 months ago

Hey @ekatef , I will try tomorrow and inform you with the outcome, thank you for fixing the error. Apart from this error, I am also unable to run any pypsa-earth related rule such as generating renewable profiles with WSL. May be the other errors are corrected by your fixes too.

Could you share the errors? a student has recently run everything with the WSL smoothly, we are happy to support :)

FYI: @Margherita-Capitani has found similar issues and solved. In particular, she is implementing also documentation to help clarify how to use pypsa-distribution to a different region. That is not well documented and may be the reason for your error. For example, build_natura_raster shall be disabled when using the default configuration (tutorial:false) or any country beside NG/BJ/MA/BW (the ones supported by the tutorials in pypsa-earth).

PR: https://github.com/pypsa-meets-earth/pypsa-distribution/pull/39

Emre-Yorat89 commented 6 months ago

I have updated my pypsa-distribution fork and tried to run snakemake -j 1 resources/renewable_profiles/profile_solar.nc to execute a pypsa-earth workflow. The result is missing input files which should be downloaded automatically in normal case I believe. Maybe I need to check how pypsa-earth workflow is working which configuration file it uses then try again.

renewable_generation_issue

Margherita-Capitani commented 6 months ago

Hi @Emre-Yorat89 , I don't know if you are running a different configuration than the default (Nigeria and Benin). If so, I have encountered problems similar to yours, you can try to see if things are better this way, take this advice of mine with a grain of salt as I am new to pypsa-distribution.

Run the default tutorial configuration ( snakemake -j 1 solve_network) first to see that there are no further problems. When finished you should see among your files a " config.yaml" file and a " config.pypsa-earth.yaml" file ( the latter already present before the run).

Try these settings:

countries -> enter the code of the nation you want to simulate
tutorial -> if the country falls within the cases that can be simulated with the tutorial, leave tutorial true, if the country you want to model does not fall within the cases that can be modeled with the tutorial set tutorial to false.

In case you have set tutorial to false then you will have to search for "cutout-2013-era5-tutorial" and replace it with "cutout-2013-era5" in all the positions where it appears ( you can do this easily with the search and replace command)

In the enable section set:

retrieve_databundle: true retrieve_cost_data: true download_osm_data: true build_cutout: false build_nature_raster: false

All These operations must be performed for both configuration files ( config.yaml and config.pypsa-warth.yaml).

Finally, in the config.yaml file, enter the coordinates of the vertices of the rectangle area you want covered by your microgrid ( you can get them from the OpenStreetMap web app).

Emre-Yorat89 commented 2 months ago

Hi @Margherita-Capitani , thank you for your valuable suggestions. The program now definitely tries to download the initial data. Currently, the dependencies on ramp_build_demand_profile script causes the workflow to fail therefore the files are not downloaded. I think when the workflow runs without error, all the necessary files will be downloaded.

ramp_module_error

pypsa-meets-earth / pypsa-distribution