Use of /queryables for filtering on feature properties without "filter" query parameter

[ycespb]

The notion of queryables and the /queryables response is currently tied to the "filtering" extension and the "filter" HTTP query parameter. However, the PGC API-Features Core (Part 1) already mentions http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_parameters_for_filtering_on_feature_properties which to be implemented without inspecting the OpenAPI definition requires also a list of "queryables" to which this filtering_on_feature_properties applies. Could there not be a separate conformance class that supports the above filtering through advertising queryables via /collections/id/queryables without requiring support for the "filter" keyword. This would allow for supporting queries such as 'function=public+use' (example from http://docs.opengeospatial.org/is/17-069r3/17-069r3.html#_parameters_for_filtering_on_feature_properties where "function" is an HTTP query parameter.

The existence of such simpler mechanism exploiting "queryables" without "filter" HTTP query parameter seems to be assumed by the draft OGC API-Records standard at https://docs.ogc.org/DRAFTS/20-004.html#_overview_4.

[cportele]

This is how we have implemented it in ldproxy - the queryables of an API with Feature resources are also mapped to query parameters as specified in Part 1.

However, in Part 1 the requirements are intentionally kept to a minimum. The query parameters are documented in the API definition, but there is no requirement to "duplicate" this information in an additional resource.

The Queryables resource is added in Part 3, because for APIs implementing richer filtering support the additional overhead to implement another resource seems acceptable and the information cannot be documented properly in the API definition anyhow.

I don't think that the Queryables resource belongs into Part 1 for those reasons. What could be an option is to make the Queryables resource a separate conformance class in Part 3 (with the implication that if Features Part 1 is supported, queryables that are feature properties must also be implemented as query parameters). The Filter conformance class would then depend on the Queryables conformance class.

But is "worth" an additional conformance class? I think most implementations implementing Queryables will also support Filtering and implementations of Part 1 are also free to support the Queryables resource in the API without support for Filtering - the Queryables would be discoverable from the Collection resource or the API definition, just not the Conformance Declaration.

[aaime]

I guess a possible attraction of this combination would be to avoid declaring the queryables in the API document to start with, in order to support implementations with many (thousands or more) collections without having to generate an impractically large API definition.

[m-mohr]

Is there a conformance class that tells a client what a server supports? For CQL (filter=...) there's a conformance class and queryables apply, but how can a client detect whether the ?prop=... notion is supported and whether queryables apply there or not?

[ycespb]

Indeed, there should be a way (e.g. a dedicated conformance class in Part 3 from which the "filter" conformance class then for instance depends) to support "queryables" (and/queryables for each individual "collection") (as current EO/CEOS OpenSearch interfaces do) without having to implement a full query language (like CQL). Otherwise, this would be an obstacle to migrate implementations from their OpenSearch API (already returning GeoJSON) to OGC API - Feature / Records / STAC implementations.

[cportele]

Meeting 2023-02-15:

• We will add two new conformance class to Part 3:
  • Queryables: The Queryables resource.
  • Queryables as query parameters: All Queryables of a collection are implemented as query parameters as specified in Part 1.
• A dependency to the Queryables conformance class will be added to the Filter conformance class.
• In addition /rec/filter/queryables-schema will be updated with using the format keyword to identify the geometry to simplify parsing for clients. This follows the approach in Processes, but with a few changes:
  • Extend the list of values to: geojson-geometry, geojson-point, geojson-multipoint, geojson-linestring, geojson-multilinestring, geojson-polygon, geojson-multipolygon, geojson-geometrycollection
  • Since JSON Schema 2019-09 and later support adjacent keywords next to a $ref member, we will simply add the format member and avoid the more complex allOf construct. Example:

    {
    "title": "Geometry",
    "description": "The geometry of the road.",
    "$ref" : "https://geojson.org/schema/LineString.json",
    "format": "geojson-geometry-linestring"
    }

@cportele will create the PR.

[ycespb]

@cportele, the new conformance class is a very elegant solution and allows for filtering without the full-fledged filter and CQL complexity. It would be nice if strict interpretation of this would allow a server to advertise via the /collections/{collectionId}/queryables response the following information to clients using the normal JSON Schema encoding:

• whether a search parameter is mandatory or optional in a search request via "required": ["datetime"] (some backends may require the presence of datetime for instance or a STAC API implementation may require the collections queryable to be present) "required": ["collections"].
• whether a standard/builltin parameter (e.g. bbox, datetime) is actually available for this collection (would require that all such parameters are included in the Queryables response otherwise clients should assume they do not apply). This is maybe less of an issue for API-Features (although we have such EO collections @ESA where {geo:box} does not apply), but in case of a STAC API Item Search, being able to advertise that intersects is not supported (due to a backend limitation) is of interest.
• whether the filter (and filter-lang) parameter is supported for this collection or not (on a collection-by-collection basis). By including this parameter as one of the parameters in the Queryables response, the server can support CQL filter for some collections and not for others. This allows for more fine-grained control instead of having a single "filter conformance class" applicable to all collections. In case of federation of heterogenous endpoints, this would be very useful.

This could allow encoding in JSON(-Schema) Queryables what in the past was encoded in an XML-based (collection-specific) OpenSearch OSDD document with Parameter extensions (without the need for the client to parse collection-specific OpenAPI documents).

[m-mohr]

• whether a search parameter is mandatory or optional in a search request via "required": ["datetime"] (some backends may require the presence of datetime for instance or a STAC API implementation may require the collections queryable to be present) "required": ["collections"].

@ycespb If this is set to required, can I still make a simple request to /items without any parameters or would datetime always be required? The latter seems like a bad idea as it would break clients that don't support queryables.

[ycespb]

@m-mohr This is a valid point, However, it you look at it from the server implementation side (in some cases a proxy in front of (multiple) legacy systems), in some cases such "open" requests are not possible or not allowed (if result sets become too large due to lack of precision of the request, typically no time or geographical constraint present). In some systems, the "indexes" are optimised per "collection" and open-requests not specifying the collection is not possible or very inefficient/slow. I would expect however that a request to /items without any parameters is easier to handle server-side than a request to /search without any parameters (collections in particular). The combination of the "Queryables" conformance class (or the "Filter" conformance class) with the STAC "/search" endpoint is not so easy in my opinion. As the "/search" is in principle cross-collection, clients would have to use the intersection of the Queryables response of all concerned collections to see what queryables are allowed at that endpoint (with which restrictions - min/max/enums) unless exactly one collection is identified in the collections queryable. The same issue exists for the use of filter in combination with /collections/collection-id/queryables responses.

[m-mohr]

I'm just talking about /items - /items without query parameters should always return a result (can be paginated and as such be limited to x random items), otherwise, many clients won't be able to handle the API.

[cportele]

@ycespb - Thank you for the comments. My thoughts:

• On collections/{collectionId}/items no query parameter is or can be be required in an implementation of OGC API Features. In general, no queryable can be required (it would imply that every filter expression must include the value).
• If a server decides that it will not execute the request it should return a 403 response:

  The 403 (Forbidden) status code indicates that the server understood the request but refuses to fulfill it. A server that wishes to make public why the request has been forbidden can describe that reason in the response content (if any)."

• A statement whether filter or filter-lang are supported for a collection is out-of-scope for the Queryables resource, which describes the queryable properties, not the API.
  • Isn't it an edge case that the conformance declaration cannot be used, because an API supports filter for some collections and not for others. I have not seen such an API, why would one declare queryables for a collection and then not support filter for that collection, but support filter on other collections? And what is the worst that can happen? The client will receive a 400 response and identify that it cannot use filter.
• /search: In the current proposal and the Testbed-18 implementation / Engineering Report, collections is required. But again, it is not a queryable, this is specified by the schema of the Search request.
• bbox and datetime have to be supported for all collections. If a collection does not have a spatial or temporal property, this will be reflected in the Collection resource:

  The member spatial only needs to be provided in the extent object, if features in the feature collection have spatial properties. The same applies to temporal and features with temporal properties. For example, a feature collection where features have a spatial, but no temporal property will only provide the spatial member.

[m-mohr]

• If a server decides that it will not execute the request it should return a 403 response:

@cportele Shouldn't that be a 400? 403 (and 401) are primarily meant for authorization: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/403

[cportele]

@m-mohr I don't think so. The text in the deprecated RFC 7231 was actually kind of implying this, but "refuses to authorize it" was changed in RFC 9110 to "refuses to fulfill it" (as it was in RFC 2616). 403 is commonly used to indicate insufficient permissions, but it is not restricted to this.

There was a discussion in the IETF HTTP WG about this topic.