LitPop's set_country method gives me a WARNING and then value error

[AMPlummer]

I am working through CLIMADA's tutorial overview section.

When I get to the part where I need to use LitPop, I get the following warning message:

2021-06-13 07:39:22,578 - climada.entity.exposures.gpw_import - WARNING - GPW data dimensions mismatch. Actual dimensions: 360 x 720
2021-06-13 07:39:22,579 - climada.entity.exposures.gpw_import - WARNING - Expected dimensions: 17400x43200 or 21600x43200

Then, I get a Value Error:

ValueError: operands could not be broadcast together with shapes (1458,) (1360,)

Here is the code I used:

from climada.entity.exposures import LitPop
exp_litpop = LitPop()
exp_litpop.set_country('Puerto Rico', res_arcsec = 120)

I imagine that the error is because I downloaded the incorrect file.

I downloaded the zip gpw-v4-population-count-rev11_2015_30_sec_tif.zip from https://sedac.ciesin.columbia.edu/data/set/gpw-v4-population-count-rev11/data-download

I then transferred the gpw-v4-population-count-rev11_2015_30_sec.tif file to the correct file path.

[tovogt]

From the file names it seems like you downloaded the correct file, but judging from the resolution, it seems like you have the file at 30 arc-minutes resolution instead of 30 arc-seconds resolution... Let's double-check: Would you please report the file size and the output of gdalinfo for the GeoTIFF-file that you downloaded? File size should be 409 MB. If it's only 343 KB, then you have the data at 30 arc-minutes resolution, which is the wrong data set.

[sameberenz]

Hi! In case the comment above does not resolve the issue, could you please check at which function & line of the code you get the ValueError? Also, which branch / release of CLIMADA are you working with? Thanks!

For a quick work-around, consider downloading the CSV file for Puerto Rico from https://www.research-collection.ethz.ch/handle/20.500.11850/331316 and reading it into an pandas DataFrame, ... then you can initiate an Exposure from this, c.f. https://climada-python.readthedocs.io/en/latest/tutorial/climada_entity_Exposures.html

[AMPlummer]

From the file names it seems like you downloaded the correct file, but judging from the resolution, it seems like you have the file at 30 arc-minutes resolution instead of 30 arc-seconds resolution... Let's double-check: Would you please report the file size and the output of gdalinfo for the GeoTIFF-file that you downloaded? File size should be 409 MB. If it's only 343 KB, then you have the data at 30 arc-minutes resolution, which is the wrong data set.

Here is the output of gdalinfo

Driver: GTiff/GeoTIFF
Files: C:\Users\plua\climada\data\gpw_v4_population_count_rev13_2015_30_sec.tif
Size is 720, 360
Coordinate System is:
GEOGCRS["WGS 84",
    DATUM["World Geodetic System 1984",
        ELLIPSOID["WGS 84",6378137,298.257223563,
            LENGTHUNIT["metre",1]]],
    PRIMEM["Greenwich",0,
        ANGLEUNIT["degree",0.0174532925199433]],
    CS[ellipsoidal,2],
        AXIS["geodetic latitude (Lat)",north,
            ORDER[1],
            ANGLEUNIT["degree",0.0174532925199433]],
        AXIS["geodetic longitude (Lon)",east,
            ORDER[2],
            ANGLEUNIT["degree",0.0174532925199433]],
    ID["EPSG",4326]]
Data axis to CRS axis mapping: 2,1
Origin = (-180.000000000000000,89.999999999999915)
Pixel Size = (0.500000000000000,-0.500000000000000)
Metadata:
  AREA_OR_POINT=Area
  DataType=Generic
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (-180.0000000,  90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)
Lower Left  (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)
Upper Right ( 180.0000000,  90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)
Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)
Center      (  -0.0000000,  -0.0000000) (  0d 0' 0.00"W,  0d 0' 0.00"S)
Band 1 Block=720x2 Type=Float32, ColorInterp=Gray
  NoData Value=-3.40282306073709653e+38
  Metadata:
    RepresentationType=ATHEMATIC

The Tif file is 395 KB but this is the 30sec resolution.

[AMPlummer]

Hi! In case the comment above does not resolve the issue, could you please check at which function & line of the code you get the ValueError? Also, which branch / release of CLIMADA are you working with? Thanks!

For a quick work-around, consider downloading the CSV file for Puerto Rico from https://www.research-collection.ethz.ch/handle/20.500.11850/331316 and reading it into an pandas DataFrame, ... then you can initiate an Exposure from this, c.f. https://climada-python.readthedocs.io/en/latest/tutorial/climada_entity_Exposures.html

Here is the full output of the Value Error

ValueError                                Traceback (most recent call last)
<ipython-input-17-847950b626c1> in <module>
----> 1 exp_litpop.set_country('Puerto Rico', res_arcsec = 120)  # We'll go lower resolution than default to keep it simple
      2 # exp_litpop.set_geometry_points() # Set geodataframe geometries from lat lon data
      3 
      4 # ent.exposures = exp_litpop
      5 

~\Desktop\climada_python-2.1.1\climada_python-2.1.1\climada\entity\exposures\litpop.py in set_country(self, countries, **args)
    200         # Get LitPop
    201         LOGGER.info('Generating LitPop data at a resolution of %s arcsec.', str(resolution))
--> 202         litpop_data = _get_litpop_box(cut_bbox, resolution, 0, reference_year,
    203                                       exponents)
    204         shp_file = shapereader.natural_earth(resolution='10m',

~\Desktop\climada_python-2.1.1\climada_python-2.1.1\climada\entity\exposures\litpop.py in _get_litpop_box(cut_bbox, resolution, return_coords, reference_year, exponents)
    399     del bm_temp
    400 
--> 401     litpop_data = _LitPop_multiply(nightlights, gpw, exponents=exponents)
    402 
    403     if return_coords == 1:

~\Desktop\climada_python-2.1.1\climada_python-2.1.1\climada\entity\exposures\litpop.py in _LitPop_multiply(nightlights, gpw, exponents)
    422     """
    423     litpop_data = pd.arrays.SparseArray(
--> 424         np.multiply(nightlights.to_numpy()**exponents[0], gpw.to_numpy()**exponents[1]), fill_value=0)
    425     return litpop_data
    426 

I get the error at line 424.

[tovogt]

The file that you have is definitely 30 arc-minutes resolution. Please download the correct file from https://sedac.ciesin.columbia.edu/downloads/data/gpw-v4/gpw-v4-population-count-rev11/gpw-v4-population-count-rev11_2015_30_sec_tif.zip

By the way, your gdalinfo-output clearly says that your file has rev13 in its name. But this version of GPW doesn't exist. How is this possible?

[AMPlummer]

The file that you have is definitely 30 arc-minutes resolution. Please download the correct file from https://sedac.ciesin.columbia.edu/downloads/data/gpw-v4/gpw-v4-population-count-rev11/gpw-v4-population-count-rev11_2015_30_sec_tif.zip

By the way, your gdalinfo-output clearly says that your file has rev13 in its name. But this version of GPW doesn't exist. How is this possible?

Thanks @tovogt . I must have changed the file name by mistake. Anyway, your suggestion worked.