juseg / hyoga

Paleoglacier modelling framework
https://hyoga.io
GNU General Public License v3.0
8 stars 0 forks source link

Grid interpolation examples fail on pandas 2.2.0. #83

Open juseg opened 3 months ago

juseg commented 3 months ago

Documentation build raises an OverflowError on ~pandas>=2.0.0~ pandas==2.2.0.

python examples/interp/plot_interpolated_surface.py

Traceback (most recent call last):
  File "conversion.pyx", line 156, in pandas._libs.tslibs.conversion.cast_from_unit_vectorized
OverflowError: value too large

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/julien/git/code/hyoga/examples/interp/plot_interpolated_surface.py", line 31, in <module>
    ds = ds.hyoga.interp(hyoga.open.example('pism.alps.vis.refined.nc'))
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/git/code/hyoga/hyoga/core/accessor.py", line 383, in interp
    ds = ds.interp(x=x, y=y)
         ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/dataset.py", line 3377, in interp
    if is_duck_dask_array(var.data):
                          ^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 434, in data
    return self.values
           ^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 607, in values
    return _as_array_or_item(self._data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/variable.py", line 313, in _as_array_or_item
    data = np.asarray(data)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 658, in __array__
    self._ensure_cached()
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 655, in _ensure_cached
    self.array = NumpyIndexingAdapter(np.asarray(self.array))
                                      ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 628, in __array__
    return np.asarray(self.array, dtype=dtype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/core/indexing.py", line 529, in __array__
    return np.asarray(array[self.key], dtype=None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/variables.py", line 72, in __array__
    return self.func(self.array)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/times.py", line 333, in decode_cf_timedelta
    result = to_timedelta_unboxed(num_timedeltas.ravel(), unit=units)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/xarray/coding/times.py", line 316, in to_timedelta_unboxed
    result = pd.to_timedelta(value, **kwargs).to_numpy()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/tools/timedeltas.py", line 213, in to_timedelta
    return _convert_listlike(arg, unit=unit, errors=errors)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/tools/timedeltas.py", line 266, in _convert_listlike
    td64arr = sequence_to_td64ns(arg, unit=unit, errors=errors, copy=False)[0]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/julien/.local/lib/python3.12/site-packages/pandas/core/arrays/timedeltas.py", line 1059, in sequence_to_td64ns
    data = cast_from_unit_vectorized(data, unit or "ns")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "conversion.pyx", line 160, in pandas._libs.tslibs.conversion.cast_from_unit_vectorized
pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: cannot convert input -775154880000.0 with the unit 's'
juseg commented 3 months ago

The issue only concerns the latest release of pandas 2.2.0 and has been documented in:

According to the trace above ds.hyoga.interp() somehow tries to decode CF times and that triggers the bug.