Sensors, measurements, observations and part 2 (geo data aka collections)

[jerstlouis]

In an attempt to offload the sensors-specific portion of an already lengthy and complex discussion of the overall "collections discussion" in #140, we can continue the discussion here.

[dblodgett-usgs]

@KathiSchleidt

Finally, on merging STA within a wider OGC-API, I do pity the poor developer confronted with an API that drastically modifies it's request structure for one specific branch of the wider API endpoints.

I pity the poor developer who has to figure out that two services offer the same data even though they have completely different landing page and top level metadata.

We're putting two incompatible technology carts in front of a single horse and expecting things to go in the right direction.

[hylkevds]

@KathiSchleidt

Finally, on merging STA within a wider OGC-API, I do pity the poor developer confronted with an API that drastically modifies it's request structure for one specific branch of the wider API endpoints.

I pity the poor developer who has to figure out that two services offer the same data even though they have completely different landing page and top level metadata.

I tend to agreed with @dblodgett-usgs here. Most clients will not be fetching data from both APIs. I think it's more important to clearly communicate that the different APIs serve the same data set, and that is easiest done by serving them from a common landing page. Ideally, clients will look at the list of conformance classes, and from the ones they recognise know which paths to use. Within the OGC-API family this already happens. A Features client won't know what to do with tiles, but that's fine, since the Features client won't request data from the Tiles paths. Similarly, a STA client will just send its requests to the STA paths, and a (theoretical future) OGC-API-Enabled OData client would only access the OData paths.

It's not so much about making sure every client will magically know how to deal with every path that is offered, and more about making sure that every client knows how to find those paths that offer the data in the way it knows.

This is of course a topic totally separate from the other topic mixed in this thread: How to offer OMS like data in a Features like API. Maybe we need to split this Issue into two separate ones?

[rob-metalinkage]

Agree we are falling over a few different things here :-) At one level it is not at all clear there is a common agreement what a building block is - @jerstlouis thanks for that pointer - certainly thats one model that covers some of the building blocks being proposed - but its a bit upside down, in the methodology is being described in the specialised extensions not the core AFAICT. (ok its evolving - and something needs to be pushed back to the core)

There are some design contracts here - in that APIs cannot have paths that clash for the composition/mix-in pattern to work - does swagger_cli check this and reject attempts to bundle clashing building blocks - at any rate the OGC infrastructure needs to check this automatically and reject building block proposals that cannot be composed this way.

Finally, its not just the client choosing the pathways - its the client knowing which pathways a particular data source is available from - for example you might do a map search to find sensors using API-features, but then access the data streams from these using STA - its still not clear to me how we can declare these are the same. The /collections path allows for links to schemas using "describedBy" relation - its perfectly possible to have different data models for the same data both described by geojson schema - so this seems weak.

is it as simple as a contract that "describedBy" has to reference a specialised schema for a distinct meta-model for each API pathway - and does this imply further restrictions on naming (and validity checking we need to do at the building block register end ?)

[hylkevds]

@joanma747: Merging the STA /v1.1 directly into the landing page may work, but would make it impossible to support different STA versions in the future. Not a big deal for /v1.0 and /v1.1, but /v2.0 is likely to be not compatible with v1.1 clients. I was thinking something along the lines of this example, having a data set that is exposed through Features, Tiles, STA 1.1 and OData:

─/ <landing page>
 ├─ conformance <conformance classes listing both Features, Tiles and high-level STA & OData conformance classes
 ├─ api <combined OpenAPI definition for everything?>
 ├─ collections <OGC-API Features, with use-case specific views on the data>
 ├─ tiles <OGC-API tiles>
 ├─ v1.1 <STA interface, with detailed STA conformance classes on the landing page>
 └─ ODATA4.01 <Full OData interface>

Would that work?

The /api document would get awfully large. Maybe we could $ref some parts and host those at v1.1/api and ODATA4.01/api to reduce the document size for clients that are only interested in the /collections or /tiles parts?

[joanma747]

@hylkevds, looks like a nice alternative to me. Using on a "version number" to identify the STA interface seems a bit odd to me so. I'll propose a small modification to it:

─/ <landing page>
 ├─ conformance <conformance classes listing both Features, Tiles and high-level STA & OData conformance classes
 ├─ api <combined OpenAPI definition for everything?>
 ├─ collections <OGC-API Features, with use-case specific views on the data>
 ├─ tiles <OGC-API tiles>
 ├─ sta/v1.1 <STA interface, with detailed STA conformance classes on the landing page>
 └─ ODATA4.01 <Full OData interface>

The /api interface does not have to describe the STA interface necessarily. Instead it should provide a collections endpoint for sensor data in GeoJSON or STAJSON (with datasteam as starting point).

[KathiSchleidt]

@hylkevds @joanma747 : any update on this approach? I'm wondering if we couldn't integrate this within some of the currently running IEs, in order to bridge the gap between the requirement for simple access to features vs. the requirements for provision of measurement data.

[dblodgett-usgs]

This was brought up as a critical need for the hydrology domain at a session today. I think the discussion above is pointing to a good candidate approach to include STA in the context of the OGC-API landing page. How can we move this forward? Is there a contribution to be made to one of the API Common artifacts?

[Sam-Bolling]

I am interested in the outcome of this discussion as well!

[rob-metalinkage]

Please note also https://github.com/opengeospatial/ogcapi-sosa which is where a binding of OMS and SOSA to the GeoJSON/FG-JSON feature schemas (API-Features compatible) is being developed.

[jerstlouis]

Since Connected Systems is taking on this aspect, let's continue the discussion in terms of whether Part 2 is suitable for integration with Connected Systems.

[rob-metalinkage]

ConnectedSystems has a different scope, that extend OMS concepts, but should be based on a canonical approach to OMS. It is however following the current sub-optimal approach encouraged by OAS 3.0, of copy and paste elements of related specifications in local copies of schemas, leaving no trace of interoperability expectations beyond descriptive text. ConnectedSystems is aiming at a OAS 3.1 baseline, which does not need to do this, however without a canonical OMS based schema this is still not transparent. Since it is based on OAS 3.1 and current versions of JSON schema it is not intended to be compatible with OGC API common in its current version, and cannot be regarded as providing a solution.

For a deeper understanding of the issues read this blog post from one of the JSON schema and OAS technical leads: https://modern-json-schema.com/your-path-from-openapi-30-to-31-and-beyond.

the OMS SWG is the relevant scope for this, and is working with the OGC/W3C Spatial Data on the Web Working Group (SDW) on an update to the SOSA ontology and has flagged work items for JSON encoding and Observable Properties descriptions. The SDW is undergoing a re-charter exercise, so any specific requirements

Re-noting, a OGC API common compatible solution is in draft form for key concerns around testing the ObservationCollections concepts as an implementation of the SOSA v2 model. (https://github.com/opengeospatial/ogcapi-sosa)

This particular issue should be decomposed into a set of relevant concerns:

firstly, the original issue description is relevant - its critically important not to just accrete partial solutions into a common, complex and unstable specification, but instead ensure the profiling and extension mechanisms are clear and tested and domain specific specialisations can be easily created with transparent and testable interoperability with the core.

secondly, transition from OAS 3.0 baseline to future versions needs to be understood and specification development processes tested to support this.

thirdly, systematically applying the lessons from the first two concerns by refactoring or updating "monolithic" specifications. Where these have been defined using "cut and paste" approach to interoperability on some aspects this may mean publishing new versions, or possibly just defining OGC BuildingBlocks to package them better, that make the interoperability intentions clear and validate all examples and artefacts to ensure the specifications meet the requirements stated.