pangeo-data / xESMF

Universal Regridder for Geospatial Data
http://xesmf.readthedocs.io/
MIT License
189 stars 34 forks source link

Fix dependencies - sparse and numba #168

Closed aulemahal closed 2 years ago

aulemahal commented 2 years ago

Fixes #158

I tried sparse 0.7.0, 0.8.0, 0.9.1 and 0.10.0 (current is 0.13.0) and only 0.7.0 made the test suite fail. I thus assumed the minimum version to be 0.8.0.

Our explicit use of numba is very superficial, I didn't really check what the minimum version would be. We could simplify the code by pinning 0.55, but that's the last minor version, it felt too soon to pin only to remove half a line.

I also updated the pre-commit hooks (there was a bug with my version of black).

aulemahal commented 2 years ago

This is super weird, but the upstream failures are because of changes in xarray. I opened an issue : pydata/xarray#6607.

So, in test_frontend.py we have test datasets with 2D lon and lat. Then we have this function that transforms them into 1D lon and lat : https://github.com/pangeo-data/xESMF/blob/9bb2d54a59cb045f71f635e8a28caf3a8e03f9a3/xesmf/tests/test_frontend.py#L281-L286

Then, in test_regrid_with_1d_grid_infer_bounds we call that function and rename x and y to lon and lat. This is a bit incomplete because if you inspect the dataset you realize that ds.lon.indexes is empty, even though ds.lon is a coordinate along a dimension of the same name. Then xesmf needs to cf_xarray to get the bounds. Taking the bounds involves the following operation: lon - lon.diff('lon'). Which fails because lon has no indexes, which are used to align the variables.

The thing is cf_xarray does call lon = lon.reset_coords(drop=True). I think this was planned to remove useless ancillary coordinates before computing the bounds. But it had the positive effect of kinda resetting lon so that the indexes were added. This is why the code worked up to the last release of xarray. A recent change in the master (I suspect "explicit indexes") broke that "workaround" and our test suite.

Ouf. That was a deep dive.

I'll push a fix for our code, I'm not sure if this is really a bug, or at least if it merits a real fix in xarray.