jerstlouis
commented
4 years ago Attached ogcapi.txt
is an example request for an OGC API combining and chaining diffent modular blocks. It would render a map combining 3 layers: satellite imagery, soil characteristics and elevation contours. It uses the following OGC API modular blocks:
-
Processes (with 'Maps' as a specialized process),
-
Coverages (a 'data access' modular block, i.e. delivering a specific type of data layer aka 'collection')
-
Features (a 'data access' modular block, i.e. delivering a specific type of data layer aka 'collection', including STAC as a specialized Features API)
-
Styles
-
Tiles (optionally)
This JSON along with BBOX, WIDTH and HEIGHT parameters can be POSTed to an OGC API implementing the "Map" process at the end of the daisy chain
It can also be used together with a Tiles API (or alternatively potentially the DGGS API) to provide an efficient mechanism to cache the results on either or both client & server.
Tiles (or DGGS, or subsetting, or other data partioning/delivery modular blocks) can also be used along the daisy chain, between the final Map process and the intermediate processes & data delivery APIs, as indpendently negotiated between the different hops of the daisy chain (invisible to the user at the end of the chain). This could potentially following the client tiles end-requests, or be driven by the hops themselves determining that to be the most efficient way to perform the requests, based on declared API conformance classes. This partitioning or chunking of the data can offer distributed computing possibility, allowing the whole process to complete in a short time synchronously.
The nested structure of this JSON document is based on the following simple types:
class URLOrID : String; // A text string identifying either a resource of particular type on the local server (if a simple identifier), or a URL to a remote server class Process : UrlOrID; // A process resource class Collection : UrlOrID; // A collection resource // (e.g. a Feature Collection, a Coverage (including ready-to display imagery or rendered map as a specialized kind of gridded coverage // whose cells are pixels) class AbstractCollection { String id; // An id allowing to name the individual inputs, e.g. useful to use as layer IDs within a style sheet Map<String, any_object> parameters; // The list of parameters to supply for retrieving this collection (e.g. query parameters after the ?) // These are parameters for either a collection end-point (e.g. Feature or Coverage) or for a process // If a data partitioning/delivery API is additional combined (e.g. tiles) (based on negotiation by the daisy chain) // these parameters should still apply Array<String, any_object> inputParameters; // The list of parameters specific to this one collection, which will be used by the process using this collection as an INPUT. // If this collection is a process, these parameters will be used with the OUTPUT of that process. } class CollectionRetrieval : AbstractCollection { Collection collection; // The collection resource } class ProcessInvokation : AbstractCollection { Process process; // The process resource Array<AbstractCollection> inputs; // The collections to use as inputs (which could themselves be ProcessInvokation to form a daisy chain) }It also attempts to encode Cascading Map Style Sheets (http://docs.opengeospatial.org/per/18-025.pdf -- Appendix C) as a JSON object to define portrayal rules directly within this document, as an alternative to referencing an external style sheet or encoding a style sheet in a single text field.
The processes involved are the following:
-
Map (local) -- The map rendering processing end-point that this request is being POSTed to)
-
Map (remote) -- A remote Map process from Landcare Research NZ
-
STACToCoverage -- A process taking as input the items returned from a STAC feature collection output, access data from where the imagery is hosted (e.g. S3 bucket), and will provide this data as a Coverage API collection
-
Sentinel2BandsToARGB -- A process creating an ARGB image out of a Sentinel2 Coverage (e.g. selecting bands, panchromatic sharpening, atmospheric correction) The output is a specialized pixels-based gridded coverage, for which a simple WM(T)S-like API might be suffcient, i.e. it can simply return a PNG image based on a BBOX or tiles for example.
-
ElevationContours -- A process generating elevation contours from a Digital Elevation Model, at a specific distance interval
Some ways this request can be used include:
-
Include additional BBOX, WIDTH and HEIGHT parameters directly in this document (or as additional query parameters for the GET), and get a PNG map back
-
POST this document to a /collections/ end-point to create a new virtual collection, to which you could then also append a /tiles/ to retrieve it as tiles
-
POST this document to a /map/tiles/ end-point and get back a templated URL for retrieving this map rendered as tiles
If the user wants to perform client-side rendering, data layers of Coverage/Imagery or Features can be requested in the same manner (skipping the Map rendering process).
More complex or resource intensive workflows might be, which this system should also support. An additional "callback" field on the top-level ProcessInvokation might be useful for asynchronous support. Similarly, estimate and billing (including chainable calculation support) can be implemented as additional fields to this request (e.g. "estimateOnly" : true). Even though the processes can (optionally?) be all linked/described at /processes/ , the actual end-point for any process could be anywhere. (Including on a separate for /processes/ to act as a catalog of process, just like /collections/ could act as a catalog of collections found elsewhere, and both could provide a search mechanism for finding useful processes & collections from all over the world, which a user might find useful).
The processes can be discoverable at /processes . A single process resource can be by itself a very versatile tool, effectively making it possible to upload code, virtual machines, containers, or algorithms as parameters, e.g.:
-
WCPS Runner
-
ADES Runner
-
R runner
-
Python runner Alternatively one could also POST to /processes to create a new process pre-baking some parameters and/or inputs.
The top level object is a
ProcessInvokation.
m-mohr
bpross-52n
One might see a process as acting on 0, 1 or multiple "collection" (data) input, possibly in addition to parameters, and outputs 0, 1 or multiple "collection" (data) outputs.
In addition to nicely tying OGC API - Processes with Features and Coverages, it makes daisy chaining very easy to accomplish.
The "data delivery" building blocks (e.g. Features & Coverages) provide the mechanisms by which to identify "collections" of vector features or grid cells (data), and retrieve them. Built-in space partitioning mechanism such as bounding boxes or more complex sub-setting can be used to efficiently retrieve part of the data, or special building blocks such as Tiles or DGGS can offer additional exchange mechanisms. This makes processes highly scalable as different tiles, DGGS cells or subsets of the data could be processed in a massively parallel manner, all through a potential chain of processes. The specific space partitioning modular API blocks (e.g. Tiles or DGGS cell), or the base Features/Coverage mechanisms can be negotiated at any exchange level throughout the chain.
Since it seems that we have to stick with the generic but loaded term of "collections" for e.g. OGC API - Features, I think it would be a mistake to overload the term of "collection" with a meaning other than a "set of geospatial data with particular characteristics", such as a generic group of things (e.g. a list of available processes). One can imagine the confusion if what OGC processes do is act on "collections", while "collection" is also used to mean a regrouping of multiple processes.
A simple example is vectorization of a digital elevation model. You have a collection representing a DEM (it is important here that the level of a "collection" should represent a "collection of gridded cells", not a collection of a collection of gridded cells, unless said collection of collections is intended to be treated as if the cells were all together in one single DEM layer). This is your input collection to a "vectorization" process. You invoke it with parameters, and you get a "vector" collection back. You can then chain another process working on a vector collection.
It can also be seen that rendering a "map" (as in the Maps API) is clearly a process rendering 1 or more "collections", taking a style as parameter, and producing an imagery ((A)RGB natural color image) "collection" as an output (the rendered map). The result can be retrieved through various data partitioning mechanism (e.g. bounding box, or tiles), and the process can be optimized to render portions as required, cached, in parallel, etc.
We think it makes a lot of sense for OGC API - Processes to support both synchronous (POST and get the 200 output result) and asynchronous modes, Maps and Routes being an important example of how this is useful for simple POST requests returning an output collection (e.g. a route or rendered map) right away.
There could be standardized "well known" processes such as the Maps Rendering process, the Routing process (open routing pilot), Vectorization process, etc, which would further greatly enhance interoperability and chainability of different processes. Most processes would also provide the output in a "collection" interface, which any other process, data delivery, or geospatial viewer supporting the OGC API would readily support. This also makes it possible to only process data on an as-needed basis, e.g. when first visualizing the output (or being requested by further down the raw data-process-visualization daisy chain) for a certain area or resolution level.
The concept of workflow could be simply described as a chain of OGC API resources and parameters. The actual communication between any 2 services in the chain can be negotiated (e.g. supporting tiles, supporting DGGS, supported data formats), and need not be included in that workflow description, so that alternate implementations with different support could apply the same workflow.
All this makes for a great universal geospatial processing framework by which all processes and data collections become readily compatible, and one can easily discover and combine data sets from all over and using processes hosted anywhere.
I brought up these ideas to the Process group at the OGC API Hackathon, as well as previously within OGC API Common discussions.
Additional background in https://github.com/opengeospatial/oapi_common/issues/17
33 #2 #35 #30