retrieve_databundle_light data download problem for new regions

[Emre-Yorat89]

Checklist

• [ ] I am using the current main branch or the latest release. Please indicate.
• [ ] I am running on an up-to-date pypsa-earth environment. Update via conda env update -f envs/environment.yaml.

Describe the Bug

snakemake -j1 retrieve_databundle_light command generates an error that is given below for Turkey. In earliest version the script were working without problem.

I have discussed with Davide about problem and his comment was "The problem is that the output files of retrieve_databundle are calculated as the unique of all output files of the data bundles and the data bundles also include the cutouts that are not however downloaded now in Turkey case. The temporary solution is: run the script using the button "run in vscode" without using snakemake then, once it finishes, you can deactivate retrieve_databundle"

Open to your suggestions to fix the problem.

Error Message

If applicable, paste any terminal output to help illustrating your problem. In some cases it may also be useful to share your list of installed packages: conda list.

<paste here>

[ekatef]

@Emre-Yorat89, thanks a lot for reporting. The issue you have mentioned is being addressed in #518 and I believe soon we'll have a general solution for it.

As a quick fix you may load the data you need directly using urls specified in configs/bundle_config.yaml, e.g. for the globally applicable data set: https://github.com/pypsa-meets-earth/pypsa-earth/blob/14f468ed75fe59e29acef13a0cbcac78e2387408/configs/bundle_config.yaml#L124-L139

[davide-f]

I think that for usability in other regions its best to fix this problem. I wanted to spare some time to fast fix it. The revision of the retrieve in #518 would take longer, so it's best to first add this fix. @ekatef if you start from a clean repo and run any region not in Africa, the problem arises. I think we need a fadt fix for usability purposes

[ekatef]

@davide-f, agree with you and Emre that it's not a particularly user-friendly behaviour to throw a puzzling error during the very first workflow step:) Do you need help with such "interim" fix?

[davide-f]

@Emre-Yorat89 A PR is pending to fix this issue, it will be merged in the next coupled of days :)