NOAA-OWP / hydrotools

Suite of tools for retrieving USGS NWIS observations and evaluating National Water Model (NWM) data.
Other
53 stars 12 forks source link

NWM Client New Test Failure: AttributeError: 'EntryPoints' object has no attribute 'get' #211

Closed aaraney closed 1 year ago

aaraney commented 1 year ago

The test runners for #210 found a failing nwm_client_new test. The AttributeError is raised in xarray source when getting a xarray backend. See the below collapsed test failure trace for more detail.

FAILED python/nwm_client_new/tests/test_NWMFileProcessor.py::test_get_dataset - AttributeError: 'EntryPoints' object has no attribute 'get'
full trace [source](https://github.com/NOAA-OWP/hydrotools/actions/runs/3500004127/jobs/5862205193#step:5:39) ```shell =================================== FAILURES =================================== _______________________________ test_get_dataset _______________________________ Unclosed client session client_session: def test_get_dataset(): > ds = NWMFileProcessor.get_dataset(input_directory) python/nwm_client_new/tests/test_NWMFileProcessor.py:8: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/hydrotools/nwm_client_new/NWMFileProcessor.py:59: in get_dataset ds = xr.open_mfdataset(file_list, engine="netcdf4") /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/xarray/backends/api.py:908: in open_mfdataset datasets = [open_(p, **open_kwargs) for p in paths] /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/xarray/backends/api.py:908: in datasets = [open_(p, **open_kwargs) for p in paths] /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/xarray/backends/api.py:[48](https://github.com/NOAA-OWP/hydrotools/actions/runs/3500004127/jobs/5862205193#step:5:49)1: in open_dataset backend = plugins.get_backend(engine) /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/xarray/backends/plugins.py:161: in get_backend engines = list_engines() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ @functools.lru_cache(maxsize=1) def list_engines(): > entrypoints = entry_points().get("xarray.backends", ()) E AttributeError: 'EntryPoints' object has no attribute 'get' /opt/hostedtoolcache/Python/3.7.15/x[64](https://github.com/NOAA-OWP/hydrotools/actions/runs/3500004127/jobs/5862205193#step:5:65)/lib/python3.7/site-packages/xarray/backends/plugins.py:105: AttributeError =============================== warnings summary =============================== python/nwm_client_new/tests/test_NWMClient.py::test_QueryError /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/hydrotools/nwm_client_new/NWMFileClient.py:197: RuntimeWarning: No data found for configuration 'analysis_assim' and reference time '20T00Z' warnings.warn(message, RuntimeWarning) python/nwm_client_new/tests/test_NWMClient.py::test_QueryError /opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/hydrotools/nwm_client_new/NWMFileClient.py:197: RuntimeWarning: No data found for configuration 'analysis_assim' and reference time '30T01Z' warnings.warn(message, RuntimeWarning) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED python/nwm_client_new/tests/test_NWMFileProcessor.py::test_get_dataset - AttributeError: 'EntryPoints' object has no attribute 'get' ```

A quick look on xarray's issue tracker led me to this -- a verbatim reproduction of this issue. Conversation there pointed to the issue's source, the newly released version (5.0.0) of importlib-metadata and python 3.7 (no longer actively supported by xarray).

aaraney commented 1 year ago

This does not appear to affect > python 3.7. I see two paths forward here, 1.) we drop support for python 3.7 for nwm_client_new, 2.) we pin the version of importlib-metadata to < 5.0.0 for python 3.7 only (I would have to look into how to do this). Thoughts, @jarq6c?

jarq6c commented 1 year ago

xarray dropping support for python 3.7 seems like a good reason for us to drop it. It looks like other downstream projects have linked to that PR and also chosen to drop 3.7 support. Python 3.7 will hit EOL by 2023-06 or 2023-07.

Probably a good move to drop support for 3.7 and bring in support for 3.10. I've begun resenting having to from typing import Dict, Tuple, List anyway.

aaraney commented 1 year ago

Im happy with dropping 3.7 support also. When did you intend to switch nwm_client and nwm_client_new? Just curious is now is a good time or at least at the next release, swap in nwm_client_new and drop support for 3.7?

aaraney commented 1 year ago

Nice, #219 is a repeat of this.

aaraney commented 1 year ago

@jarq6c, for now to stop the github actions from failing I am going to pin importlib-metadata<=4.13.0. When we get around to dropping support for 3.7 ill remove that dependency.