WaterData().bygeom() error

rmcd-mscb commented 1 year ago

What happened?

The Following code returns an InputValueError in the async_retriever package.

# USGS gage 01482100 Delaware River at Del Mem Bridge at Wilmington De
gage_id = "01482100"
nldi = NLDI()
del_basins = nldi.get_basins(gage_id)
huc12_basins = WaterData(layer="huc12").bygeom(del_basins.geometry[0])

What did you expect to happen?

No response

Minimal Complete Verifiable Example

No response

MVCE confirmation

[ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
[ ] Complete example — the example is self-contained, including all data and the text of any traceback.
[ ] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

---------------------------------------------------------------------------
InputValueError                           Traceback (most recent call last)
Cell In[10], line 1
----> 1 huc12_basins = WaterData(layer="huc12").bygeom(del_basins.geometry[0])

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pynhd/pynhd.py:457, in WaterData.bygeom(self, geometry, geo_crs, xy, predicate, sort_attr)
    415 def bygeom(
    416     self,
    417     geometry: Polygon | MultiPolygon,
   (...)
    421     sort_attr: str | None = None,
    422 ) -> gpd.GeoDataFrame:
    423     """Get features within a geometry.
    424 
    425     Parameters
   (...)
    455         The requested features in the given geometry.
    456     """
--> 457     resp = self.wfs.getfeature_bygeom(
    458         geometry, geo_crs, always_xy=not xy, predicate=predicate, sort_attr=sort_attr
    459     )
    460     resp = cast("list[dict[str, Any]]", resp)
    461     return self._to_geodf(resp)

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pygeoogc/pygeoogc.py:646, in WFS.getfeature_bygeom(self, geometry, geo_crs, always_xy, predicate, sort_attr)
    643 if predicate.upper() not in valid_predicates:
    644     raise InputValueError("predicate", valid_predicates)
--> 646 return self.getfeature_byfilter(
    647     f"{predicate.upper()}({geom_name}, {g_wkt})", method="POST", sort_attr=sort_attr
    648 )

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/pygeoogc/pygeoogc.py:735, in WFS.getfeature_byfilter(self, cql_filter, method, sort_attr)
    733 else:
    734     headers = {"content-type": "application/x-www-form-urlencoded"}
--> 735     resp = ar.retrieve_text([self.url], [{"data": payload, "headers": headers}], "POST")
    736 try:
    737     nfeatures = int(resp[0].split(self.nfeat_key)[-1].split(" ")[0].strip('"'))

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/async_retriever.py:498, in retrieve_text(urls, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    437 def retrieve_text(
    438     urls: Sequence[StrOrURL],
    439     request_kwds: Sequence[dict[str, Any]] | None = None,
   (...)
    447     raise_status: bool = True,
    448 ) -> list[str]:
    449     r"""Send async requests and get the response as ``text``.
    450 
    451     Parameters
   (...)
    496     '01646500'
    497     """
--> 498     return retrieve(
    499         urls,
    500         "text",
    501         request_kwds,
    502         request_method,
    503         max_workers,
    504         cache_name,
    505         timeout,
    506         expire_after,
    507         ssl,
    508         disable,
    509         raise_status,
    510     )

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/async_retriever.py:394, in retrieve(urls, read_method, request_kwds, request_method, max_workers, cache_name, timeout, expire_after, ssl, disable, raise_status)
    330 def retrieve(
    331     urls: Sequence[StrOrURL],
    332     read_method: Literal["text", "json", "binary"],
   (...)
    341     raise_status: bool = True,
    342 ) -> RESPONSE:
    343     r"""Send async requests.
    344 
    345     Parameters
   (...)
    392     '01646500'
    393     """
--> 394     inp = BaseRetriever(
    395         urls,
    396         read_method=read_method,
    397         request_kwds=request_kwds,
    398         request_method=request_method,
    399         cache_name=cache_name,
    400     )
    402     if not disable:
    403         disable = os.getenv("HYRIVER_CACHE_DISABLE", "false").lower() == "true"

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/_utils.py:175, in BaseRetriever.__init__(self, urls, file_paths, read_method, request_kwds, request_method, cache_name)
    170     self.read_method = "read" if read_method == "binary" else read_method
    171     self.r_kwds = (
    172         {"content_type": None, "loads": json.loads} if read_method == "json" else {}
    173     )
--> 175 self.url_kwds = self.generate_requests(urls, request_kwds, self.file_paths)
    177 self.cache_name = create_cachefile(cache_name)

File ~/mambaforge/envs/gdptools-examples/lib/python3.10/site-packages/async_retriever/_utils.py:208, in BaseRetriever.generate_requests(urls, request_kwds, file_paths)
    206 if not_found:
    207     invalids = ", ".join(not_found)
--> 208     raise InputValueError(f"request_kwds ({invalids})", list(session_kwds))
    210 return zip(url_id, urls, request_kwds)

InputValueError: Given request_kwds (data, headers) is invalid. Valid options are:
self
method
str_or_url
expire_after
kwargs

Anything else we need to know?

I tried with versions 0.15.0 and 0.15.1.

Wondering if it was related to Margaux's issue I also tried:

huc12_basins = WaterData(layer="huc12").bygeom(del_basins.reset_index().geometry[0].simplify(1e-3))

Environment

Conda environment:

name: "gdptools-examples"
channels:
  - numba
  - bokeh
  - plotly
  - conda-forge
  - nodefaults
dependencies:
  - python>=3.9,<=3.10
  - pip
  - pydantic<=1.10.9
  - gdptools==0.2.3
  - netcdf4<=1.6.0
  - owslib
  - jupyterlab
  - jupytext
  # HyRiver packages
  - pynhd==0.15.0
  - pygeoogc==0.15.0
  - pygeohydro==0.15.0
  - bottleneck
  - numba
  - pyogrio
  - cartopy
  - hvplot
  - geoviews
  - holoviews
  - bokeh
  - datashader
  - networkx
  - flask
  - selenium
  - geckodriver
  - firefox
  - plotly
  - plotly-geo
  - nodejs

cheginit commented 1 year ago

Rich, thanks for reporting the issue. There was an issue with a breaking change from one of the dependencies of async-retriever. I resolved the issue, but still haven't released a new version. Please update async-retriever from git and see if it solves your issue.

pip install git+https://github.com/hyriver/async-retriever

rmcd-mscb commented 1 year ago

@cheginit That worked Taher, could you please tag me here when the new package is available? Thank You!

cheginit commented 1 year ago

Sure! Most probably, I will release new versions of HyRiver over the weekend.

rmcd-mscb commented 1 year ago

Ok - thanks Taher. I'll close this and look for new version early next week. Much appreciated!

hyriver / pynhd