pypsa-meets-earth / pypsa-distribution

GNU General Public License v3.0
10 stars 8 forks source link

Population raster not found #18

Closed ekatef closed 11 months ago

ekatef commented 12 months ago

Checklist

Describe the Bug

When changing country setting from ["NG"] to ["KZ"], an error appears in bulid demand complaining that there is no a raster with population data.

It looks like get_WorldPop_path assumes that the population raster is always available in pypsa-earth repo, which can be not the case. My feeling is it makes sense to add an option to load the file in case it doesn't exist.

Error Message

A full listing looks like follow:

[Sun Oct  8 00:20:21 2023]
rule build_demand:
    input: data/sample_profile.csv, networks/base.nc, resources/shapes/microgrid_shapes.geojson, resources/buildings/cluster_with_buildings.geojson
    output: resources/demand/microgrid_load.csv
    log: logs/build_demand.log
    jobid: 0
    benchmark: benchmarks/build_demand
    reason: Input files updated by another job: resources/buildings/cluster_with_buildings.geojson, networks/base.nc
    resources: tmpdir=/var/folders/qn/vpndfm21795ckkq89np1ckp40000gn/T, mem_mb=3000, mem_mib=2862

~/opt/miniconda3/envs/pypsa-earth-upd16/lib/python3.10/site-packages/pypsa/networkclustering.py:16: UserWarning: The namespace `pypsa.networkclustering` is deprecated and will be removed in PyPSA v0.24. Please use `pypsa.clustering.spatial instead`. 
  warnings.warn(
INFO:pypsa.io:Imported network base.nc has buses, lines
INFO:rasterio._env:GDAL signalled an error: err_no=4, msg='~/Documents/_github_/pypsa-distribution/pypsa-earth/data/WorldPop/kaz_ppp_2020_UNadj_constrained.tif: No such file or directory'
Traceback (most recent call last):
  File "rasterio/_base.pyx", line 310, in rasterio._base.DatasetBase.__init__
  File "rasterio/_base.pyx", line 221, in rasterio._base.open_dataset
  File "rasterio/_err.pyx", line 221, in rasterio._err.exc_wrap_pointer
rasterio._err.CPLE_OpenFailedError: ~/Documents/_github_/pypsa-distribution/pypsa-earth/data/WorldPop/kaz_ppp_2020_UNadj_constrained.tif: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/Documents/_github_/pypsa-distribution/.snakemake/scripts/tmpfhyk_5hq.build_demand.py", line 216, in <module>
    estimate_microgrid_population(
  File "~/Documents/_github_/pypsa-distribution/.snakemake/scripts/tmpfhyk_5hq.build_demand.py", line 102, in estimate_microgrid_population
    with rasterio.open(raster_path) as src:
  File "~/opt/miniconda3/envs/pypsa-earth-upd16/lib/python3.10/site-packages/rasterio/env.py", line 451, in wrapper
    return f(*args, **kwds)
  File "~/opt/miniconda3/envs/pypsa-earth-upd16/lib/python3.10/site-packages/rasterio/__init__.py", line 304, in open
    dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
  File "rasterio/_base.pyx", line 312, in rasterio._base.DatasetBase.__init__
rasterio.errors.RasterioIOError: ~/Documents/_github_/pypsa-distribution/pypsa-earth/data/WorldPop/kaz_ppp_2020_UNadj_constrained.tif: No such file or directory
[Sun Oct  8 00:20:24 2023]
Error in rule build_demand:
    jobid: 0
    input: data/sample_profile.csv, networks/base.nc, resources/shapes/microgrid_shapes.geojson, resources/buildings/cluster_with_buildings.geojson
    output: resources/demand/microgrid_load.csv
    log: logs/build_demand.log (check log file(s) for error details)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-10-08T002013.122300.snakemake.log