search

ESA-VirES / VirES-Python-Client

viresclient is a Python package for easy access to Swarm & Aeolus products as xarray.Dataset
https://viresclient.readthedocs.io
MIT License
17 stars 1 forks source link

to_netcdf can fail due to usage of CategoricalDtype #112

Open smithara opened 3 months ago

smithara commented 3 months ago

Current versions of (pandas/xarray?) are now causing a failure in xarray.Dataset.to_netcdf with the datasets generated by viresclient where "Spacecraft" is stored as a pandas.Categorical, where this didn't happen before. https://github.com/ESA-VirES/VirES-Python-Client/blob/f15fa07e204bfc27e86c50026b106cc2641f7fa2/src/viresclient/_data_handling.py#L238

For example:

import pandas as pd
import xarray as xr

ds = xr.Dataset(
    data_vars={
        "x": ("t", pd.Categorical(["A", "B", "B", "A"], categories=["A", "B"]))
    }
)
ds.to_netcdf("test.h5")

fails with:

TypeError: Cannot interpret 'CategoricalDtype(categories=['A', 'B'], ordered=False, categories_dtype=object)' as a data type
smithara commented 3 months ago

Workaround: Downgrade to xarray 2024.3.0