WaterData bounding box error #78

Closed ehinman closed 1 month ago

What happened?

I have a CI pipeline in GitLab that builds Jupyter notebooks into documentation pages using Sphinx. One of the notebooks has the following line: huc_shapes = WaterData('huc08').bybox([-115.065380, 45.947037, -112.692334, 47.572536])

The CI pipeline fails on this line with the following error: AttributeError: 'CachedResponse' object has no attribute '__aexit__'

What did you expect to happen?

I expected the notebook to run the WaterData cell with no issues and build the documentation in html pages. It works on my local version of the repository, but it fails in the CI pipeline on GitLab.

Minimal Complete Verifiable Example

No response

MVCE confirmation

[ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
[ ] Complete example — the example is self-contained, including all data and the text of any traceback.
[ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

Notebook error:
CellExecutionError in examples/regional_runoff_calculations.nblink:
------------------
huc_shapes = WaterData('huc08').bybox([-115.065380, 45.947037, -112.692334, 47.572536])
------------------
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 huc_shapes = WaterData('huc08').bybox([-115.065380, 45.947037, -112.692334, 47.572536])
File /usr/local/lib/python3.11/site-packages/pynhd/pynhd.py:633, in WaterData.bybox(self, bbox, box_crs, sort_attr)
    610 def bybox(
    611     self,
    612     bbox: tuple[float, float, float, float],
    613     box_crs: CRSTYPE = 4326,
    614     sort_attr: str | None = None,
    615 ) -> gpd.GeoDataFrame:
    616     """Get features within a bounding box.
    617 
    618     Parameters
   (...)
    631         The requested features in a GeoDataFrames.
    632     """
--> 633     resp = self.wfs.getfeature_bybox(
    634         bbox,
    635         box_crs,
    636         always_xy=True,
    637         sort_attr=sort_attr,
    638     )
    639     resp = cast("list[dict[str, Any]]", resp)
    640     return self._to_geodf(resp)
File /usr/local/lib/python3.11/site-packages/pygeoogc/pygeoogc.py:586, in WFS.getfeature_bybox(self, bbox, box_crs, always_xy, sort_attr)
    575 bbox_str = f'{",".join(str(round(c, 6)) for c in bbox)},{box_crs.to_string()}'
    576 payload = {
    577     "service": "wfs",
    578     "version": self.version,
   (...)
    584     "resultType": "hits",
    585 }
--> 586 resp = ar.retrieve_text([self.url], [{"params": payload}])
    587 try:
    588     nfeatures = int(resp[0].split(self.nfeat_key)[-1].split(" ")[0].strip('"'))
File /usr/local/lib/python3.11/site-packages/async_retriever/async_retriever.py:500, in retrieve_text(urls, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    439 def retrieve_text(
    440     urls: Sequence[StrOrURL],
    441     request_kwds: Sequence[dict[str, Any]] | None = None,
   (...)
    449     raise_status: bool = True,
    450 ) -> list[str]:
    451     r"""Send async requests and get the response as ``text``.
    452 
    453     Parameters
   (...)
    498     '01646500'
    499     """
--> 500     return retrieve(
    501         urls,
    502         "text",
    503         request_kwds,
    504         request_method,
    505         max_workers,
    506         cache_name,
    507         timeout,
    508         expire_after,
    509         ssl,
    510         disable,
    511         raise_status,
    512     )
File /usr/local/lib/python3.11/site-packages/async_retriever/async_retriever.py:433, in retrieve(urls, read_method, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    430 loop, new_loop = utils.get_event_loop()
    431 results = (loop.run_until_complete(session(url_kwds=c)) for c in chunked_reqs)
--> 433 resp = [r for _, r in sorted(tlz.concat(results))]
    434 if new_loop:
    435     loop.close()
File /usr/local/lib/python3.11/site-packages/async_retriever/async_retriever.py:431, in <genexpr>(.0)
    429 chunked_reqs = tlz.partition_all(max_workers, inp.url_kwds)
    430 loop, new_loop = utils.get_event_loop()
--> 431 results = (loop.run_until_complete(session(url_kwds=c)) for c in chunked_reqs)
    433 resp = [r for _, r in sorted(tlz.concat(results))]
    434 if new_loop:
File /usr/local/lib/python3.11/site-packages/nest_asyncio.py:98, in _patch_loop.<locals>.run_until_complete(self, future)
     95 if not f.done():
     96     raise RuntimeError(
     97         'Event loop stopped before Future completed.')
---> 98 return f.result()
File /usr/local/lib/python3.11/asyncio/futures.py:203, in Future.result(self)
    201 self.__log_traceback = False
    202 if self._exception is not None:
--> 203     raise self._exception.with_traceback(self._exception_tb)
    204 return self._result
File /usr/local/lib/python3.11/asyncio/tasks.py:277, in Task.__step(***failed resolving arguments***)
    273 try:
    274     if exc is None:
    275         # We use the `send` method directly, because coroutines
    276         # don't have `__iter__` and `__next__` methods.
--> 277         result = coro.send(None)
    278     else:
    279         result = coro.throw(exc)
File /usr/local/lib/python3.11/site-packages/async_retriever/async_retriever.py:236, in async_session_with_cache(url_kwds, read, r_kwds, request_method, cache_name, timeout, expire_after, ssl, raise_status)
    231 request_func = getattr(session, request_method.lower())
    232 tasks = (
    233     utils.retriever(uid, url, kwds, request_func, read, r_kwds, raise_status)
    234     for uid, url, kwds in url_kwds
    235 )
--> 236 return await asyncio.gather(*tasks)
File /usr/local/lib/python3.11/asyncio/tasks.py:349, in Task.__wakeup(self, future)
    347 def __wakeup(self, future):
    348     try:
--> 349         future.result()
    350     except BaseException as exc:
    351         # This may also be a cancellation.
    352         self.__step(exc)
File /usr/local/lib/python3.11/asyncio/tasks.py:277, in Task.__step(***failed resolving arguments***)
    273 try:
    274     if exc is None:
    275         # We use the `send` method directly, because coroutines
    276         # don't have `__iter__` and `__next__` methods.
--> 277         result = coro.send(None)
    278     else:
    279         result = coro.throw(exc)
File /usr/local/lib/python3.11/site-packages/async_retriever/_utils.py:81, in retriever(uid, url, s_kwds, session, read_type, r_kwds, raise_status)
     45 async def retriever(
     46     uid: int,
     47     url: StrOrURL,
   (...)
     52     raise_status: bool,
     53 ) -> tuple[int, str | Awaitable[str | bytes | dict[str, Any]] | None]:
     54     """Create an async request and return the response as binary.
     55 
     56     Parameters
   (...)
     79         The retrieved response as binary.
     80     """
---> 81     async with session(url, **s_kwds) as response:
     82         try:
     83             return uid, await getattr(response, read_type)(**r_kwds)
File /usr/local/lib/python3.11/site-packages/aiohttp/client.py:1364, in _BaseRequestContextManager.__aexit__(self, exc_type, exc, tb)
   1358 async def __aexit__(
   1359     self,
   1360     exc_type: Optional[Type[BaseException]],
   1361     exc: Optional[BaseException],
   1362     tb: Optional[TracebackType],
   1363 ) -> None:
-> 1364     await self._resp.__aexit__(exc_type, exc, tb)
AttributeError: 'CachedResponse' object has no attribute '__aexit__'
You can ignore this error by setting the following in conf.py:
    nbsphinx_allow_errors = True
make: *** [Makefile:26: test] Error 2
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: exit code 1

Anything else we need to know?

This is the docker image we're using to run the CI pipeline: image: ${ARTIFACTORY_URL}/docker-official-mirror/python:3.11

I suspect this is a version conflict problem somewhere in the packages being used by pynhd with the docker image, but I'm not sure. Thanks!

Environment

pynhd-0.17.1

Thanks for reporting this. It seems the issue is caused by aiohttp-client-cache. It's being addressed soon. Once it's done, updating aiohttp-client-cache should fix the issue. I will then release a new version to pin the dep on aiohttp-client-cache to the fixed version.

BTW, until it gets fixed, as a workaround, you can disable caching by:

import os

os.environ["HYRIVER_CACHE_DISABLE"] = "true"

This should fix the issue at the expense of missing caching.

Thanks so much @cheginit!

@ehinman This issue is fixed with HyRiver 0.18. Please install the new version and if you still get this error, reopen this issue.

hyriver / pynhd