diff of cftime.Datetime

mathause commented 1 year ago

What happened?

A cftime variable returns a timedelta64[ns] when calling diff / + / - and it can then not be added/ subtracted from the original data.

What did you expect to happen?

We can add cftime timedeltas.

Minimal Complete Verifiable Example

import xarray as xr

air = xr.tutorial.open_dataset("air_temperature", use_cftime=True)

air.time + air.time.diff("time") / 2

MVCE confirmation

[x] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[x] Complete example — the example is self-contained, including all data and the text of any traceback.
[x] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[x] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

air.time.variable.values[1:] - air.time.variable.values[:-1]

returns array([datetime.timedelta(seconds=21600), ...]) but then

xr.Variable(("time",), np.array([datetime.timedelta(0)]))

returns a dtype='timedelta64[ns]' array.

Anything else we need to know?

See upstream PR: xarray-contrib/cf-xarray#441
Similar to #7381 (but I don't think it's the same issue, feel free to close if you disagree)
That might need a special data type for timedeltas of cftime.Datetime objects, or allowing to add 'timedelta64[ns]' to cftime.Datetime objects
The casting comes from

https://github.com/pydata/xarray/blob/d8ec3a3f6b02a8b941b484b3d254537af84b5fde/xarray/core/variable.py#L366

https://github.com/pydata/xarray/blob/d8ec3a3f6b02a8b941b484b3d254537af84b5fde/xarray/core/variable.py#L272

Environment

INSTALLED VERSIONS ------------------ commit: d8ec3a3f6b02a8b941b484b3d254537af84b5fde python: 3.10.9 | packaged by conda-forge | (main, Feb 2 2023, 20:20:04) [GCC 11.3.0] python-bits: 64 OS: Linux OS-release: 5.14.21-150400.24.63-default machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: en_GB.UTF-8 LOCALE: ('en_US', 'UTF-8') libhdf5: 1.12.2 libnetcdf: 4.9.1 xarray: 2023.2.1.dev20+g06a87062 pandas: 1.5.3 numpy: 1.23.5 scipy: 1.10.1 netCDF4: 1.6.2 pydap: installed h5netcdf: 1.1.0 h5py: 3.8.0 Nio: None zarr: 2.13.6 cftime: 1.6.2 nc_time_axis: 1.4.1 PseudoNetCDF: 3.2.2 iris: 3.4.1 bottleneck: 1.3.6 dask: 2023.2.1 distributed: 2023.2.1 matplotlib: 3.7.0 cartopy: 0.21.1 seaborn: 0.12.2 numbagg: 0.2.2 fsspec: 2023.1.0 cupy: None pint: 0.20.1 sparse: 0.14.0 flox: 0.6.8 numpy_groupies: 0.9.20 setuptools: 67.4.0 pip: 23.0.1 conda: None pytest: 7.2.1 mypy: None IPython: 8.11.0 sphinx: None

mathause commented 1 year ago

cc @spencerkclark @znicholls

znichollscr commented 1 year ago

cc @spencerkclark @znicholls

Legend thanks

larsbuntemeyer commented 1 year ago

@mathause

That might need a special data type for timedeltas of cftime.Datetime objects, or allowing to add 'timedelta64[ns]' to cftime.Datetime objects

I think this might be related:

https://github.com/numpy/numpy/issues/15479#issuecomment-1399292370

pydata / xarray