NLDI input error "x, y, z, and time must be same size"

qideng7 commented 1 year ago

What happened?

I was playing with the pynhd Quick Start examples but encountered with input error, "x, y, z, and time must be same size". It seems to be caused by crs transform.

What did you expect to happen?

No response

Minimal Complete Verifiable Example

from pynhd import NLDI, WaterData, NHDPlusHR
import pynhd as nhd
nldi = NLDI()
station_id = "01031500"
basin = nldi.get_basins(station_id)

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output

ProjError                                 Traceback (most recent call last)
Cell In[5], line 3
      1 nldi = NLDI()
      2 station_id = "01031500"
----> 3 basin = nldi.get_basins(station_id)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pynhd\pynhd.py:900, in NLDI.get_basins(self, feature_ids, fsource, split_catchment, simplified)
    898 urls = (("linked-data", fsource, fid, f"basin?{query}") for fid in feature_ids)
    899 index, resp = self._get_urls(urls, True)
--> 900 basins = geoutils.json2geodf(resp, 4269, 4326)  # type: ignore
    901 basins.index = pd.Index([feature_ids[i] for i in index], name="identifier")
    902 basins = basins[~basins.geometry.isnull()].copy()

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pygeoutils\pygeoutils.py:120, in json2geodf(content, in_crs, crs)
    118     geodf = geodf.set_crs(in_crs)
    119     if in_crs != crs:
--> 120         geodf = geodf.to_crs(crs)
    121 geodf = cast("gpd.GeoDataFrame", geodf)
    122 return geodf

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geodataframe.py:1364, in GeoDataFrame.to_crs(self, crs, epsg, inplace)
   1362 else:
   1363     df = self.copy()
-> 1364 geom = df.geometry.to_crs(crs=crs, epsg=epsg)
   1365 df.geometry = geom
   1366 if not inplace:

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geoseries.py:1124, in GeoSeries.to_crs(self, crs, epsg)
   1047 def to_crs(self, crs=None, epsg=None):
   1048     """Returns a ``GeoSeries`` with all geometries transformed to a new
   1049     coordinate reference system.
   1050 
   (...)
   1121 
   1122     """
   1123     return GeoSeries(
-> 1124         self.values.to_crs(crs=crs, epsg=epsg), index=self.index, name=self.name
   1125     )

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\array.py:779, in GeometryArray.to_crs(self, crs, epsg)
    775     return self
    777 transformer = Transformer.from_crs(self.crs, crs, always_xy=True)
--> 779 new_data = vectorized.transform(self.data, transformer.transform)
    780 return GeometryArray(new_data, crs=crs)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\_vectorized.py:1114, in transform(data, func)
   1111 result[~has_z] = set_coordinates(data[~has_z].copy(), np.array(new_coords_z).T)
   1113 coords_z = get_coordinates(data[has_z], include_z=True)
-> 1114 new_coords_z = func(coords_z[:, 0], coords_z[:, 1], coords_z[:, 2])
   1115 result[has_z] = set_coordinates(data[has_z].copy(), np.array(new_coords_z).T)
   1117 return result

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pyproj\transformer.py:430, in Transformer.transform(self, xx, yy, zz, tt, radians, errcheck, direction)
    428     intime = None
    429 # call pj_transform.  inx,iny,inz buffers modified in place.
--> 430 self._transformer._transform(
    431     inx,
    432     iny,
    433     inz=inz,
    434     intime=intime,
    435     direction=direction,
    436     radians=radians,
    437     errcheck=errcheck,
    438 )
    439 # if inputs were lists, tuples or floats, convert back.
    440 outx = _convertback(xisfloat, xislist, xistuple, inx)

File pyproj/_transformer.pyx:459, in pyproj._transformer._Transformer._transform()

ProjError: x, y, z, and time must be same size

Anything else we need to know?

No response

Environment

`SYS INFO -------- commit: None python: 3.9.16 (main, Mar 8 2023, 10:39:24) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 7, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: English_United States.1252 libhdf5: 1.10.6 libnetcdf: 4.6.1 PACKAGE VERSION ------------------------------- aiodns N/A aiohttp 3.8.3 aiohttp-client-cache 0.8.1 aiosqlite 0.18.0 async-retriever 0.14.0 bottleneck 1.3.5 brotli N/A click 8.0.4 cytoolz 0.12.0 dask 2023.4.1 defusedxml 0.7.1 folium 0.14.0 geopandas 0.12.2 h5netcdf 1.1.0 hydrosignatures 0.14.0 lxml 4.9.2 matplotlib 3.7.1 netCDF4 1.5.7 networkx 2.8.4 numba 0.57.0 numpy 1.24.3 owslib 0.29.1 pandas 1.5.3 py3dep N/A pyarrow 11.0.0 pydaymet N/A pygeohydro 0.14.0 pygeoogc 0.14.0 pygeos N/A pygeoutils 0.14.0 pynhd 0.14.0 pynldas2 N/A pyproj 2.6.1.post1 pytest 7.3.1 pytest-cov N/A rasterio 1.2.10 requests 2.29.0 requests-cache 1.0.1 richdem N/A rioxarray 0.14.1 scipy 1.10.1 shapely 2.0.1 tables 3.8.0 ujson 5.4.0 urllib3 1.26.15 xarray 2022.11.0 xdist N/A yaml N/A -------------------------------`

cheginit commented 1 year ago

Thanks for reporting the issue.

However, I cannot reproduce this issue with the latest version, 0.15. Can you update your pynhd version and try again?

qideng7 commented 1 year ago

Thanks for reporting the issue.

However, I cannot reproduce this issue with the latest version, 0.15. Can you update your pynhd version and try again? Tried with 0.15.0 version pynhd, got the same error.

from pynhd import NLDI
import pynhd

print(pynhd.__version__)

0.15.0

nldi = NLDI()
station_id = "01031500"

basin = nldi.get_basins(station_id)

ProjError Traceback (most recent call last) Cell In[4], line 9 6 nldi = NLDI() 7 station_id = "01031500" ----> 9 basin = nldi.get_basins(station_id)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pynhd\pynhd.py:900, in NLDI.get_basins(self, feature_ids, fsource, split_catchment, simplified) 898 urls = (("linked-data", fsource, fid, f"basin?{query}") for fid in feature_ids) 899 index, resp = self._get_urls(urls, True) --> 900 basins = geoutils.json2geodf(resp, 4269, 4326) # type: ignore 901 basins.index = pd.Index([feature_ids[i] for i in index], name="identifier") 902 basins = basins[~basins.geometry.isnull()].copy()

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pygeoutils\pygeoutils.py:120, in json2geodf(content, in_crs, crs) 118 geodf = geodf.set_crs(in_crs) 119 if in_crs != crs: --> 120 geodf = geodf.to_crs(crs) 121 geodf = cast("gpd.GeoDataFrame", geodf) 122 return geodf

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geodataframe.py:1364, in GeoDataFrame.to_crs(self, crs, epsg, inplace) 1362 else: 1363 df = self.copy() -> 1364 geom = df.geometry.to_crs(crs=crs, epsg=epsg) 1365 df.geometry = geom 1366 if not inplace:

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\geoseries.py:1124, in GeoSeries.to_crs(self, crs, epsg) 1047 def to_crs(self, crs=None, epsg=None): 1048 """Returns a GeoSeries with all geometries transformed to a new 1049 coordinate reference system. 1050 (...) 1121 1122 """ 1123 return GeoSeries( -> 1124 self.values.to_crs(crs=crs, epsg=epsg), index=self.index, name=self.name 1125 )

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas\array.py:779, in GeometryArray.to_crs(self, crs, epsg) 775 return self 777 transformer = Transformer.from_crs(self.crs, crs, always_xy=True) --> 779 new_data = vectorized.transform(self.data, transformer.transform) 780 return GeometryArray(new_data, crs=crs)

File ~\anaconda3\envs\flow_Ml\lib\site-packages\geopandas_vectorized.py:1114, in transform(data, func) 1111 result[~has_z] = set_coordinates(data[~has_z].copy(), np.array(new_coords_z).T) 1113 coords_z = get_coordinates(data[has_z], include_z=True) -> 1114 new_coords_z = func(coords_z[:, 0], coords_z[:, 1], coords_z[:, 2]) 1115 result[has_z] = set_coordinates(data[has_z].copy(), np.array(new_coords_z).T) 1117 return result

File ~\anaconda3\envs\flow_Ml\lib\site-packages\pyproj\transformer.py:430, in Transformer.transform(self, xx, yy, zz, tt, radians, errcheck, direction) 428 intime = None 429 # call pj_transform. inx,iny,inz buffers modified in place. --> 430 self._transformer._transform( 431 inx, 432 iny, 433 inz=inz, 434 intime=intime, 435 direction=direction, 436 radians=radians, 437 errcheck=errcheck, 438 ) 439 # if inputs were lists, tuples or floats, convert back. 440 outx = _convertback(xisfloat, xislist, xistuple, inx)

File pyproj/_transformer.pyx:459, in pyproj._transformer._Transformer._transform()

ProjError: x, y, z, and time must be same size

cheginit commented 1 year ago

Thanks! My next guess is that there's an issue either with pyproj or geopandas versions. First, let's check pyproj. In my environment that works without issue, pyproj version is 3.5 and yours is 2.6. Please only update pyproj and see if it solves the issue.

cheginit commented 1 year ago

Feel free to reopen if you still have issue with this.

hyriver / pynhd