pydata / xarray

N-D labeled arrays and datasets in Python
https://xarray.dev
Apache License 2.0
3.62k stars 1.08k forks source link

Xarray ignores the underlying unit of "datetime64" types. #6591

Open alvarosg opened 2 years ago

alvarosg commented 2 years ago

What happened?

xarray always converts "datetime64" types to nanoseconds.

xarray also silently does nothing when calling .astype('datetime64[ms]') on an datetime64 type.

What did you expect to happen?

I would expect the same behavior as in numpy (respect units, as shown in the reproducer). Or at least raise an error, when calling astype with a different unit.

Minimal Complete Verifiable Example

import numpy as np
import xarray as xa

# As expected.
time_array = np.array(["2000-01-01"]).astype('datetime64[ms]')
print(time_array.dtype)  # datetime64[ms]
print(time_array.astype('datetime64[s]').dtype)  # datetime64[s]

# Always end up with "ns".
time_data_array = xa.DataArray(data=time_array, dims=("time"))
print(time_data_array.dtype)  # datetime64[ns]
print(time_data_array.astype('datetime64[s]').dtype)  # datetime64[ns]


### MVCE confirmation

- [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
- [X] Complete example — the example is self-contained, including all data and the text of any traceback.
- [X] Verifiable example — the example copy & pastes into an IPython prompt or [Binder notebook](https://mybinder.org/v2/gh/pydata/xarray/main?urlpath=lab/tree/doc/examples/blank_template.ipynb), returning the result.
- [X] New issue — a search of GitHub Issues suggests this is not a duplicate.

### Relevant log output

_No response_

### Anything else we need to know?

_No response_

### Environment

<details>

INSTALLED VERSIONS
------------------
commit: None
python: 3.7.13 (default, Apr 24 2022, 01:04:09) 
[GCC 7.5.0]
python-bits: 64
OS: Linux
OS-release: 5.4.188+
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.0
libnetcdf: 4.7.4

xarray: 0.18.2
pandas: 1.3.5
numpy: 1.21.6
scipy: 1.4.1
netCDF4: 1.5.8
pydap: None
h5netcdf: None
h5py: 3.1.0
Nio: None
zarr: None
cftime: 1.6.0
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: 1.3.4
dask: 2.12.0
distributed: 1.25.3
matplotlib: 3.2.2
cartopy: None
seaborn: 0.11.2
numbagg: None
pint: None
setuptools: 57.4.0
pip: 21.1.3
conda: None
pytest: 3.6.4
IPython: 5.5.0
sphinx: 1.8.6

</details>
dcherian commented 2 years ago

I agree. We should document this and raise an error.

SimonHeybrock commented 2 years ago

Note duplicate (or related): #5750