[Bug]: xESMF related issue "AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead."

[tomvothecoder]

What happened?

xESMF is conflicting with numpy>=2.0, which is breaking the GitHub Actions build (here).

I opened an issue in the xESMF repo here: https://github.com/pangeo-data/xESMF/issues/370

What did you expect to happen? Are there are possible answers you came across?

Related code in xESMF: https://github.com/pangeo-data/xESMF/blob/530b804c28a9b4f64dd360384897c4c2b34ab8c3/xesmf/smm.py#L229

• [x] Update conda envs to constrain numpy <2.0
• [x] Update conda-forge feedstock recipe with same constraint (link)

Minimal Complete Verifiable Example (MVCE)

No response

Relevant log output

=================================== FAILURES ===================================
________________________ TestXESMFRegridder.test_regrid ________________________

self = <tests.test_regrid.TestXESMFRegridder object at 0x7f78dd5a7190>

    def test_regrid(self):
        ds = self.ds.copy()

        regridder = xesmf.XESMFRegridder(ds, self.new_grid, "bilinear")

>       output = regridder.horizontal("ts", ds)

tests/test_regrid.py:734: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
xcdat/regridder/xesmf.py:158: in horizontal
    regridder = xe.Regridder(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:928: in __init__
    super().__init__(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:385: in __init__
    self.weights = add_nans_to_weights(self.weights)
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/smm.py:229: in add_nans_to_weights
    m.data[krow] = [np.NaN] if m.data[krow] == [] else m.data[krow]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

attr = 'NaN'

    def __getattr__(attr):
        # Warn for expired attributes
        import warnings

        if attr == "linalg":
            import numpy.linalg as linalg
            return linalg
        elif attr == "fft":
            import numpy.fft as fft
            return fft
        elif attr == "dtypes":
            import numpy.dtypes as dtypes
            return dtypes
        elif attr == "random":
            import numpy.random as random
            return random
        elif attr == "polynomial":
            import numpy.polynomial as polynomial
            return polynomial
        elif attr == "ma":
            import numpy.ma as ma
            return ma
        elif attr == "ctypeslib":
            import numpy.ctypeslib as ctypeslib
            return ctypeslib
        elif attr == "exceptions":
            import numpy.exceptions as exceptions
            return exceptions
        elif attr == "testing":
            import numpy.testing as testing
            return testing
        elif attr == "matlib":
            import numpy.matlib as matlib
            return matlib
        elif attr == "f2py":
            import numpy.f2py as f2py
            return f2py
        elif attr == "typing":
            import numpy.typing as typing
            return typing
        elif attr == "rec":
            import numpy.rec as rec
            return rec
        elif attr == "char":
            import numpy.char as char
            return char
        elif attr == "array_api":
            raise AttributeError("`numpy.array_api` is not available from "
                                 "numpy 2.0 onwards")
        elif attr == "core":
            import numpy.core as core
            return core
        elif attr == "strings":
            import numpy.strings as strings
            return strings
        elif attr == "distutils":
            if 'distutils' in __numpy_submodules__:
                import numpy.distutils as distutils
                return distutils
            else:
                raise AttributeError("`numpy.distutils` is not available from "
                                     "Python 3.12 onwards")

        if attr in __future_scalars__:
            # And future warnings for those that will change, but also give
            # the AttributeError
            warnings.warn(
                f"In the future `np.{attr}` will be defined as the "
                "corresponding NumPy scalar.", FutureWarning, stacklevel=2)

        if attr in __former_attrs__:
            raise AttributeError(__former_attrs__[attr])

        if attr in __expired_attributes__:
>           raise AttributeError(
                f"`np.{attr}` was removed in the NumPy 2.0 release. "
                f"{__expired_attributes__[attr]}"
            )
E           AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead.

/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/numpy/__init__.py:397: AttributeError
___________________ TestXESMFRegridder.test_preserve_bounds ____________________

self = <tests.test_regrid.TestXESMFRegridder object at 0x7f78dd3af7d0>

    def test_preserve_bounds(self):
        ds = fixtures.generate_dataset(
            decode_times=True, cf_compliant=False, has_bounds=True
        )

        ds = ds.drop_vars(["lat_bnds", "lon_bnds"])

        regridder = xesmf.XESMFRegridder(ds, self.new_grid, method="bilinear")

>       output = regridder.horizontal("ts", ds)

tests/test_regrid.py:793: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
xcdat/regridder/xesmf.py:158: in horizontal
    regridder = xe.Regridder(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:928: in __init__
    super().__init__(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:385: in __init__
    self.weights = add_nans_to_weights(self.weights)
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/smm.py:229: in add_nans_to_weights
    m.data[krow] = [np.NaN] if m.data[krow] == [] else m.data[krow]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

attr = 'NaN'

    def __getattr__(attr):
        # Warn for expired attributes
        import warnings

        if attr == "linalg":
            import numpy.linalg as linalg
            return linalg
        elif attr == "fft":
            import numpy.fft as fft
            return fft
        elif attr == "dtypes":
            import numpy.dtypes as dtypes
            return dtypes
        elif attr == "random":
            import numpy.random as random
            return random
        elif attr == "polynomial":
            import numpy.polynomial as polynomial
            return polynomial
        elif attr == "ma":
            import numpy.ma as ma
            return ma
        elif attr == "ctypeslib":
            import numpy.ctypeslib as ctypeslib
            return ctypeslib
        elif attr == "exceptions":
            import numpy.exceptions as exceptions
            return exceptions
        elif attr == "testing":
            import numpy.testing as testing
            return testing
        elif attr == "matlib":
            import numpy.matlib as matlib
            return matlib
        elif attr == "f2py":
            import numpy.f2py as f2py
            return f2py
        elif attr == "typing":
            import numpy.typing as typing
            return typing
        elif attr == "rec":
            import numpy.rec as rec
            return rec
        elif attr == "char":
            import numpy.char as char
            return char
        elif attr == "array_api":
            raise AttributeError("`numpy.array_api` is not available from "
                                 "numpy 2.0 onwards")
        elif attr == "core":
            import numpy.core as core
            return core
        elif attr == "strings":
            import numpy.strings as strings
            return strings
        elif attr == "distutils":
            if 'distutils' in __numpy_submodules__:
                import numpy.distutils as distutils
                return distutils
            else:
                raise AttributeError("`numpy.distutils` is not available from "
                                     "Python 3.12 onwards")

        if attr in __future_scalars__:
            # And future warnings for those that will change, but also give
            # the AttributeError
            warnings.warn(
                f"In the future `np.{attr}` will be defined as the "
                "corresponding NumPy scalar.", FutureWarning, stacklevel=2)

        if attr in __former_attrs__:
            raise AttributeError(__former_attrs__[attr])

        if attr in __expired_attributes__:
>           raise AttributeError(
                f"`np.{attr}` was removed in the NumPy 2.0 release. "
                f"{__expired_attributes__[attr]}"
            )
Warning:  yaksa: 10 leaked handle pool objects
E           AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead.

/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/numpy/__init__.py:397: AttributeError
_________________________ TestAccessor.test_horizontal _________________________

self = <tests.test_regrid.TestAccessor object at 0x7f78dd3f4ed0>

    def test_horizontal(self):
        output_grid = grid.create_gaussian_grid(32)

>       output_data = self.horizontal_ds.regridder.horizontal(
            "ts", output_grid, tool="xesmf", method="bilinear"
        )

tests/test_regrid.py:1161: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
xcdat/regridder/accessor.py:205: in horizontal
    output_ds = regridder.horizontal(data_var, self._ds)
xcdat/regridder/xesmf.py:158: in horizontal
    regridder = xe.Regridder(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:928: in __init__
    super().__init__(
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/frontend.py:385: in __init__
    self.weights = add_nans_to_weights(self.weights)
/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/xesmf/smm.py:229: in add_nans_to_weights
    m.data[krow] = [np.NaN] if m.data[krow] == [] else m.data[krow]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

attr = 'NaN'

    def __getattr__(attr):
        # Warn for expired attributes
        import warnings

        if attr == "linalg":
            import numpy.linalg as linalg
            return linalg
        elif attr == "fft":
            import numpy.fft as fft
            return fft
        elif attr == "dtypes":
            import numpy.dtypes as dtypes
            return dtypes
        elif attr == "random":
            import numpy.random as random
            return random
        elif attr == "polynomial":
            import numpy.polynomial as polynomial
            return polynomial
        elif attr == "ma":
            import numpy.ma as ma
            return ma
        elif attr == "ctypeslib":
            import numpy.ctypeslib as ctypeslib
            return ctypeslib
        elif attr == "exceptions":
            import numpy.exceptions as exceptions
            return exceptions
        elif attr == "testing":
            import numpy.testing as testing
            return testing
        elif attr == "matlib":
            import numpy.matlib as matlib
            return matlib
        elif attr == "f2py":
            import numpy.f2py as f2py
            return f2py
        elif attr == "typing":
            import numpy.typing as typing
            return typing
        elif attr == "rec":
            import numpy.rec as rec
            return rec
        elif attr == "char":
            import numpy.char as char
            return char
        elif attr == "array_api":
            raise AttributeError("`numpy.array_api` is not available from "
                                 "numpy 2.0 onwards")
        elif attr == "core":
            import numpy.core as core
            return core
        elif attr == "strings":
            import numpy.strings as strings
            return strings
        elif attr == "distutils":
            if 'distutils' in __numpy_submodules__:
                import numpy.distutils as distutils
                return distutils
            else:
                raise AttributeError("`numpy.distutils` is not available from "
                                     "Python 3.12 onwards")

        if attr in __future_scalars__:
            # And future warnings for those that will change, but also give
            # the AttributeError
            warnings.warn(
                f"In the future `np.{attr}` will be defined as the "
                "corresponding NumPy scalar.", FutureWarning, stacklevel=2)

        if attr in __former_attrs__:
            raise AttributeError(__former_attrs__[attr])

        if attr in __expired_attributes__:
>           raise AttributeError(
                f"`np.{attr}` was removed in the NumPy 2.0 release. "
                f"{__expired_attributes__[attr]}"
            )
E           AttributeError: `np.NaN` was removed in the NumPy 2.0 release. Use `np.nan` instead.

/usr/share/miniconda3/envs/xcdat_ci/lib/python3.11/site-packages/numpy/__init__.py:397: AttributeError

Anything else we need to know?

No response

Environment

Latest xCDAT, numpy>=2.0 and xesmf=0.8.5

[tomvothecoder]

This issue was recently fixed in xESMF by https://github.com/pangeo-data/xESMF/pull/373. We need to wait for a new release of xesmf for this to be carried over, although we still might want to constrain numpy <2.0.0 for at least another year maybe?