WFS Support

[arenger]

Hello Again,

This is a helpful plugin, and I've got it work fine with WMS. However, I'm curious: Was this plugin/extension meant to also work WFS? As a test, I loaded a dataset in three different ways: in a postgis instance, in elasticsearch, and a shapefile uploaded to the GeoServer... all the same data, served via an instance of GeoServer 2.12.0 running in Docker. I then issued six different queries, all with the same bounding box:

1) WMS to the postgis layer 2) WFS to the postgis layer 3) WMS to the shapefile layer 4) WFS to the shapefile layer 5) WMS to the elasticsearch layer 6) WFS to the elasticsearch layer

The first five queries return without error and all return the same feature count. The 6th query fails with the following error.

geoserver_1  | 21 Dec 16:38:27 ERROR [data.elasticsearch] - method [POST], host [http://es:9200], URI [/mvum/FeatureCollection/_search], status line [HTTP/1.1 500 Internal Server Error]
...
geoserver_1  | Caused by: java.io.IOException: Error executing query search
geoserver_1  |  at mil.nga.giat.data.elasticsearch.ElasticFeatureSource.getReaderInternal(ElasticFeatureSource.java:132)
geoserver_1  |  at org.geotools.data.store.ContentFeatureSource.getReader(ContentFeatureSource.java:647)
geoserver_1  |  at org.geotools.data.store.ContentFeatureCollection.features(ContentFeatureCollection.java:173)
geoserver_1  |  ... 114 more
geoserver_1  | Caused by: org.elasticsearch.client.ResponseException: method [POST], host [http://es:9200], URI [/mvum/FeatureCollection/_search], status line [HTTP/1.1 500 Internal Server Error]
geoserver_1  | {"error":{"root_cause":[{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [1000000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"mvum","node":"ve0jbu0DTiK1Iv256fIY-A","reason":{"type":"query_phase_execution_exception","reason":"Result window is too large, from + size must be less than or equal to: [10000] but was [1000000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting."}}]},"status":500}

I tried shrinking the bounding box, such that only two features are returned... but the same error occurs. Do you have any suggestions?

[sjudeng]

Try configuring logging to view request data. Monitor the logs when making the problematic request and note the request object.

27 Dec 11:38:55 DEBUG [data.elasticsearch] - Elasticsearch request:
{
  "size" : 100,
  "query" : {
    "bool" : {
      "must" : {
        "match_all" : { }
      },
      "filter" : {
        "geo_bounding_box" : {
          "geo" : {
            "top_left" : [ -7.119140625, 55.634765625 ],
            "bottom_right" : [ 31.201171875, -12.568359375 ]
          }
        }
      }
    }
  },
  "from" : 0
}

If the issue is not apparent in the request data you can post it directly against your Elasticsearch cluster to debug further.

$ curl -H "Content-Type: application/json" http://localhost:9200/status_s3/_search?pretty -d '
{
  "size" : 100,
  "query" : {
    "bool" : {
      "must" : {
        "match_all" : { }
      },
      "filter" : {
        "geo_bounding_box" : {
          "geo" : {
            "top_left" : [ -7.119140625, 55.634765625 ],
            "bottom_right" : [ 31.201171875, -12.568359375 ]
          }
        }
      }
    }
  },
  "from" : 0
}
'

If you do need to handle requests returning a large number of features try setting the scroll_enabled store parameter.

[sjudeng]

I'm suspecting you'll find the size in your request object is too large, likely because the maxFeatures parameter is unset or set too large in your GetFeature WFS request. Try setting maxFeatures to something smaller.

http://localhost:8080/geoserver/test/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=test:active&maxFeatures=10000

For GeoServer WMS requests the default maxFeatures is Integer.MAX_VALUE but for WFS requests the default is 1000000 (configurable). The plugin has a store parameter (default_max_features) that is used when maxFeatures is Integer.MAX_VALUE, which is why you don't observe the issue in WMS requests.

[arenger]

I finally got back to testing this with the scroll_enabled parameter set -- and this resolved the problem. Thank you again for your help, @sjudeng .