bird-house / flyingpigeon

WPS processes for climate model data, indices and extreme events
http://flyingpigeon.readthedocs.io/en/latest/
Apache License 2.0
19 stars 15 forks source link

Failure to use WFS subsetting #287

Closed Zeitsperre closed 5 years ago

Zeitsperre commented 5 years ago

I'm trying to run the subset_wfs_polygon process on some NetCDFs from a locally running FlyingPigeon instance but I keep running into the same errors:

Traceback (most recent call last):
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/subset_base.py", line 119, in parse_feature
    geoms = make_geoms(feature, mosaic)
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/subset_base.py", line 62, in make_geoms
    feature['crs']['properties']['name']).code
TypeError: 'NoneType' object is not subscriptable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tjs/git/flyingpigeon/src/pywps/pywps/app/Process.py", line 208, in _run_process
    self.handler(wps_request, wps_response)  # the user must update the wps_response.
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/wps_subset_wfs_polygon.py", line 65, in _handler
    geoms = self.parse_feature(request)
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/subset_base.py", line 125, in parse_feature
    raise Exception(msg)
Exception: Failed to fetch features.
geoserver: https://boreas.ouranos.ca/geoserver/wfs 
typename: public:USGS_HydroBASINS_lake_na_lev12
features ['96929, 96930']
'NoneType' object is not subscriptable

The method I'm applying to try to run this process is roughly as follows:

url = 'http://localhost:8093/wps'
geoserver = 'https://<servername>/geoserver/wfs'
nc = <locally-stored netcdf path> || <URI to netcdf served via THREDDS> # Not sure if this matters
fp = WPSClient(url, progress=True)
x = fp.subset_wfs_polygon(resource=nc, typename='public:USGS_HydroBASINS_lake_na_lev12',
                          geoserver=geoserver, featureids='96929, 96930', mosaic='False')

There doesn't seem to be any tests for this function as of yet, but I imagine there is a way of triggering it properly. I know that WFS requests using 2.0.0 protocol don't work well in PyWPS (I've employed workarounds using requests and lxlml to format a request to GeoServer in Raven), and I get the impression that the algorithm in subset_base.py expects a particular data structure (feature['crs']['properties']['name']) that I'm certain is not standard within the table of features I'm trying to grab (e.g. crs is not a field).

Is there an obvious reason why this process keeps failing?

huard commented 5 years ago

Try passing featureids as a list of strings.

Zeitsperre commented 5 years ago

Passing along ['USGS_HydroBASINS_lake_na_lev12.94396', 'USGS_HydroBASINS_lake_na_lev12.94436', 'USGS_HydroBASINS_lake_na_lev12.94437'] yields:

Traceback (most recent call last):
  File "/home/tjs/git/flyingpigeon/src/pywps/pywps/app/Process.py", line 208, in _run_process
    self.handler(wps_request, wps_response)  # the user must update the wps_response.
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/wps_subset_wfs_polygon.py", line 70, in _handler
    for res in self.parse_resources(request):
  File "/home/tjs/git/flyingpigeon/flyingpigeon/processes/subset_base.py", line 104, in parse_resources
    yield path
UnboundLocalError: local variable 'path' referenced before assignment

While passing [94396, 94436, 94437] yields the same errors as above. Getting somewhere!

huard commented 5 years ago

Please check with branch fix-287

huard commented 5 years ago

@Zeitsperre Ping

Zeitsperre commented 5 years ago

Confirmed. Your changes fix the issue!