search

PyPSA / atlite

Atlite: A Lightweight Python Package for Calculating Renewable Power Potentials and Time Series
https://atlite.readthedocs.io
255 stars 87 forks source link

Error cannot convert float nan to int #282

Closed davide-f closed 1 year ago

davide-f commented 1 year ago

Description

When using atlite to create time series for New Zealand, the following error message is thrown for offwind calculations.

Error Message

Traceback (most recent call last):
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/atlite/gis.py", line 537, in _process_func
    return shape_availability_reprojected(shapes.loc[[i]], *args)[0]
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/atlite/gis.py", line 512, in shape_availability_reprojected
    masked, transform = shape_availability(geometry, excluder)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/atlite/gis.py", line 452, in shape_availability
    masked, transform = projected_mask(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/atlite/gis.py", line 366, in projected_mask
    masked, transform_ = mask(raster, geom, crop=True, **kwargs)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/rasterio/mask.py", line 178, in mask
    shape_mask, transform, window = raster_geometry_mask(
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/rasterio/mask.py", line 80, in raster_geometry_mask
    window = geometry_window(dataset, shapes, pad_x=pad_x, pad_y=pad_y)
  File "/home/davidef/miniconda3/envs/pypsa-earth/lib/python3.8/site-packages/rasterio/features.py", line 460, in geometry_window
    row_start, row_stop = int(math.floor(min(rows))), int(math.ceil(max(rows)))
ValueError: cannot convert float NaN to integer
"""

Your Environment

euronion commented 1 year ago

Can't reproduce, sorry. Closing for now. If you encounter it again feel free to re-open the issue.

davide-f commented 1 year ago

Hello! Actually it appeared again using https://github.com/pypsa-meets-earth/pypsa-earth/pull/701 on NZ... I may have to create again the cutout

euronion commented 1 year ago

Is it the same cutout with which you encounter the problem in the first place? Or already a new one?

davide-f commented 1 year ago

Again many thanks for the quick reply! :)

Indeed it is the same cutout. I would do other checks and let you know. In particular, the tests may be:

  1. Recreate the cutout for the whole earth again
  2. Create a NZ-specific cutout All of them using the latest version of the repository.

@euronion could you confirm that some compression is already included in the new version?

Ah, as a little comment: in conda/pip the version .11 is available bit no .11 tag is available in github

euronion commented 1 year ago

Compression is included in the 0.2.11 version, yes. But don't expect significant gains unless you stitched together the world cutout by hand before. ERA5 data is already nicely compressed.

@FabianHofmann did the last release; I can see a tag, but no GitHub release. @FabianHofmann could you check?

FabianHofmann commented 1 year ago

@euronion @davide-f I made a new GH release for 2.1. Most likely not related but note @davide-f that #242 (fixing nan's in hydro calculation) did not make into this release. On the other hand, #289 should fix this mentioned issue. Did you make sure version 0.2.11 is installed?

davide-f commented 1 year ago

@FabianHofmann yeah, I confirm that 0.2.11 is installed. I also recently re-created the whole cutout with the new version and the error persists unfortunately

euronion commented 1 year ago

Can you reproduce the issue with an MWE? Or only inside the workflow?

davide-f commented 1 year ago

This problem with a custom environment is not appearing. It may be related to xarray: with the v2023 some errors in the workflow have disappeared, not sure why. Now NZ is working. This problems may be related for some reason with xarray or another package

euronion commented 1 year ago

Alright, let's keep an eye out for it. If we can track it down again it would be nice to have an test in the CI.