Sensors, measurements, observations and part 2 (geo data aka collections)

[jerstlouis]

In an attempt to offload the sensors-specific portion of an already lengthy and complex discussion of the overall "collections discussion" in #140, we can continue the discussion here.

[jerstlouis]

Based on the discussion so far in aforementioned issue, there seems to be two suggested approaches to mapping a SensorThings API to collection(s):

1. A direct mapping of the entity types, where each entity type is its own collection, and relations are established between them. (e.g. /collections/observations, /collections/sensors, /collections/featuresOfInterest)

2. A single collection integrating information from multiple entity types (e.g. Observations, which do not have a geospatial component, as properties, and FeaturesOfInterest, which have a geospatial component to define the geometry aspect). An example collection /collections/weather could provide:

   • a SensorThings API at /collections/weather/sensorThings/v1.0/
   • a Features access mode where /items is a collection of multiple point features, where each feature represents one weather station, the geometry represents the location of that weather station, reported weather observations are stored as properties, and the time of the observation is also a property
   • a Coverage access mode at /collections/weather/coverage, with different planes representing different observations, the spatial coordinates representing the location of the different stations, and the temporal coordinates representing the time of the observations.

   Alternatively, that could be split into separate collections (e.g. humidity, pressure, temperature), and each coverage would have a single plane, and the feature collection would have only the individual observed properties. In this case the SensorThings API would map best to a higher up parent collection, which reinforces the argument for hierarchical collection support (#11). Features and Coverages could even be available at both levels, organized slightly differently.

I see the first approach as targeted to new clients and services being developed wishing to implement an OGC API - Features mechanism to retrieve sensor data, while keeping to the familiar SensorThings data model. While this should totally be possible if organizations wish to do so, I wonder what the advantages are over using the existing SensorThings API?

The second approach I would myself encourage is intended as a bridge between the IoT and GIS world, for existing GIS clients to perform visualization and analysis. A link to the actual full SensorThings API is provided from that collection for SensorThings clients, which was really designed for sensor data and best handles the real-time nature and large quantity of information. But the data is also alternatively provided as a feature collection and/or a coverage, so that GIS clients can readily and easily visualize and operate on the collection of sensor information as a whole.

If there are proponents of both approaches, then I believe both should be possible ways of integrating sensor information within an OGC API.

[KathiSchleidt]

Continuing off of @dblodgett-usgs comment on ObservationCollection, should also align with what we've been doing on the ObservationCollection in O&M V3

[jeffharrison]

@KathiSchleidt

Could you please point me at 'O&M v3'?

FYI, I helped start/design Sensor Web Enablement (SWE) and 'O&M v1' ... but that was a long while ago.

Thanks and Regards, Jeff

[KathiSchleidt]

O&M V3 Work here: https://github.com/opengeospatial/om-swg Weekly SWG Calls Wed 10 CEST

[jeffharrison]

Thanks. Reading 'Rename Sensor to Observer' and other topics...

[rob-metalinkage]

There should be an obvious binding to the SOSA ontology as an expression of the O&M data model for people wishing to use JSON-LD or other semantically explicit approaches.

[jeffharrison]

Since the community has well known encodings then an OGC API should be able to implement them. This reminds me of one of the Routing APIs and the Route Exchange Model.

If you had to sketch a minimalist OGC API with just 1/2 cup of coffee ;-) one might start with a reference to O&M and some Requirements classes such as...

1. The server SHALL support the HTTP GET operation at the path /observations/{observationId} for each observation referenced from the observations resource at /observations.

2a. A successful execution of the operation SHALL be reported as a response with a HTTP status code 200.

2b. The content of that response SHALL conform to a requirements class of the Observations Model.

type: properties: type: type: type:

2c. By default (and this requirements class provides no mechanism to change the default), the content SHALL conform to the requirements class "Observations Model (full)".

2d. If the request included an Accept-Language header, the server SHALL try to honor the request and otherwise fall back to an available language.

2e. At this point you realize the observation may not be able to represented as GeoJSON, what you just wrote is really dumb and that you need more coffee...

Of course, I didn't use collections but you could (e.g. /collections/observations )

[KathiSchleidt]

@jeffharrison Sorry for the added confusion, but we had some interesting discussions on how to split O&M V3 into requirements classes during today's SWG call.

It's to do with the work on interfaces we've been doing in V3 allowing for much more abstraction if required. Led to the question of what blocks do we define as individual conformance classes for O&M. While very much a discussion in progress, it looks like we may be seriously disaggregating O&M, providing individual conformance classes for Observation, ObservableProperty, Process...

I don't quite get the bit in 2e except for the needing more coffee ;) To my current understanding while collections may contain spatial features, there is no requirement that they do. Thus, the not being able to provide all collection contents as GeoJSON is a wider issue

[KathiSchleidt]

Also - some links to the current UML (we didn't set up a SVN due to issues with the parallel ISO work, all improvised on GitHub). Main Diagrams as follow:

Abstract Observation Overview

Basic Observations Overview

Observation Core

[hylkevds]

Going back to @jerstlouis' large post with the two models.

One advantage I mentioned in the 140 discussion. Using this, it would be possible to combine STA with other feature collections in the same API. having collections for buildings, floors, rooms, vehicles, etc, and have all those seamlessly link to observations. The other way to achieve this would be to allow other entity types in the STA model: v1.1/buildings, etc.

The second options sounds like it is nice for generic GIS client, but in reality it doesn't work except in the simplest use case: each Datastream has only one Observation. Lets look at an example: our AirQuality ad-hoc, since it is one of the simpler use cases:

Data: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1 Viewer: https://api4inspire.k8s.ilt-dmz.iosb.fraunhofer.de/servlet/is/127/

• All Things are stations
• Each station has a fixed location
• Each FeatureOfInterest is the location of a station (this is already a simplification from the official model)
• Stations have up to 9 Datastreams for the ObservedProperties measured there
• Most Datastreams have several years of Observations, 1 per hour

A generic GIS client would have no problem displaying the Things/Locations or the FeaturesOfInterest, but what would you put in the properties? It's not feasible to just list all Observations for all Datastreams for each Thing; Way to much data. But every other selection is just arbitrary and will only fit specific use cases.

For generic GIS clients, we just create a query that returns GeoJSON with just the data I want to display, like:

https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Things
  ?$orderby=id asc
  &$filter=properties/countryCode eq 'DK'
  &$expand=Locations($select=location;$top=1),Datastreams(
    $select=unitOfMeasurement,name;
    $expand=ObservedProperty($select=name),Observations(
      $select=result,phenomenonTime;
      $orderby=phenomenonTime desc;
      $top=1))
  &$resultFormat=GeoJSON

Even if you want to display a certain time interval, like the data for the last day, you can do that with just a static URL. So a generic GIS client doesn't need to know SensorThings. And in my opinion, OGC Api isn't done until it is possible to do something like this in OGC Api too.

So I wouldn't focus too much on generic GIS clients, they won't be able to do much with the non-geo data any way.

[jerstlouis]

@hylkevds In my approach 2), you would be merging all observations from a single location & time in a single feature, regardless of how much data that is. And as a whole a collection represents all location and the entire duration for which observations have been collected. And Common methods for filtering by space & time are used to filter the data of interest by the regular GIS client, so I disagree with your statement that "it doesn't work except in the simplest use case", the complex use cases are what I am targeting. And in fact implementing this Features and/or Coverage bridge could be querying the SensorThings API behind the scene, much like you just showed.

As for the other way around for adding entity types to SensorThings, could there also be a way to establish links from SensorThings entities to e.g. items of a Features API, where both can be used together for what they do best?

So I wouldn't focus too much on generic GIS clients, they won't be able to do much with the non-geo data any way.

I believe they can do a lot if the data is organized as usual Feature collections or Coverages are.

[hylkevds]

you would be merging all observations from a single location & time in a single feature

Are you sure you mean a single feature ? How would that work? GetFeature would return GBs of data! You'd need ways to filter the contents of the feature... And you'd have to know what you can filter on for each feature before getting the feature... And it would not just need filtering on space and time, but also other properties, least of which is ObservedProperty. Can you give some concrete example of how this would work?

[jerstlouis]

@hylkevds all observations from a single location and time, that is a limited set of information, at least for a single feature, is it not?

Without any spatial or temporal filtering, yes it would be GBs of data, but at worst the limit/next would kick in. GBs of data are a good thing if you want to perform interesting visualization and analysis.

And of course you have the filtering. In Features, queryables and a filtering language (e.g. CQL) provide a mechanism to filter by data attributes (including a way to do do temporal filtering). And spatial partitioning like DGGS or Tiles or BBOX allows you to filter on a specific geospatial region. You can also use properties= to select which data attributes (observations) you are specifically interested in. And this GIS bridge can simply map those queries to a SensorThings API query.

I would very much like to give you a very concrete example of this, including a full live demonstration of the concept in action, but this will take a bit of time. In fact IoT / GIS integration is one major topic of our collaborative project that we just started this week! I would be happy to keep exchanging with you on this and use cases, and share our progress.

[hylkevds]

Ah, ok, a single location and time. But would that mean "duplicating" the feature for each observation made on it? Even if that feature is not just a point? Fetching a time series for one feature would mean fetching the "same" feature thousands of times... And you'd always get all measurements for that feature, not just the one you're interested in. In the water domain there are some 1500 odd common ObservedProperties...

And what do you do with all the other data? Duplicate the full ObservedProperty data for each Observation? And all Sensor metadata?

And what about the case where the UltimateFeatureOfInterest and ProximateFeatureOfInterest are different? One is the river, the other the water sample from that river that the measurement was made on.

[jerstlouis]

@hylkevds In a Features API, yes it would mean duplicating the geometry for each observation. However, this would likely be a lot less data than the amount of observation data, especially in the common case of a point, so that would be a small overhead. In Coverage API, the data model is different, and the overhead might be different (based on the type of coverage, e.g. gridded coverage).

I am curious about sensors use cases where the feature is not just a point? Is that geometry ever dynamic in those cases? Even though a feature of interest might be a more complex geometry, potentially a sensor is situated at a specific location, and perhaps a separate collection with the more complex geometry could still be associated with it with some relational mechanism.

As I mentioned in previous post, there is also a mechanism in Features to select which properties you're interested in (properties=), if you are only interested in a subset of the observations.

Sensor metadata could also be available as separately available information, as it is not essential for it to be integrated with the geospatial aspect for the visualization & analytics use cases.

The key here is that your observations and your geospatial features are integrated in a way allowing a typical GIS client to perform some interesting visualization and analytics, and leveraging the common spatio-temporal filtering mechanism (e.g. of the Features and Coverage API), while still having a real fully capable and efficient SensorThings API behind it all.

[jeffharrison]

Reading through the comments on Features API, observations and filtering. At first you might think it's a bit strange, and unworkable. However, I've helped design, develop and deploy 'REST' Feature services that did similar things with massive amounts of near real time data. Filters worked great.

Not advocating for anything, just providing input from deployments.

Best Regards, Jeff

[hylkevds]

Ah, I think I'm beginning to understand what your ideas are. That would work indeed.

For some domains it's actually not that far off from certain access patterns. In the water domain for instance, the FeatureOfInterest is the water sample, taken at a point location, with about 15 to 20 Observations made on that sample. So there already exists a FoI for each phenomenonTime and location combination. The "sample view" of this dataset is more of less what you describe. Example service at BRGM: https://sensorthings-wq.brgm-rec.fr/FROST-Server/v1.0/

It probably would still push quite a bit more data over the line than needed, depending on the exact domain model, but that's not always a problem.

I didn't know Features had the "properties=" query option, good that they have that too. I love it in STA.

[hylkevds]

Another relevant topic is that of nested collections.

The SensorThings API has the entity sets ("collections")

• /Things
• /Datastreams

But also:

• /Things(<thingId>)/Datastreams

One could see that as a nested collection, but in reality it is not. It is a subset of the /Datastreams collection. An entity can not exist in such a nested entity set without being in a top-level entity set.

Of course this only works because STA (OData) has entity linking as part of the core data model.

[rob-metalinkage]

From the perspective of conformance - if you wanted to state something about a service that supported a query that delivered a result set you need three things 1) the service behaviour and query payloads 2) the returned objects (feature model) 3) the payload schema - how returned objects are bundled together as a set

we seem to keep conflating these issues, and its still not obvious to me exactly how we declare both the containing payload schema and the target feature model in a flexible way in the spirit of OGCAPI core...

can someone provide an example to show how Observation features are contained in a predictable container model, and exactly where that container model is specified (is that OGCAPI- features, and the implication is that an O&M capable profile therefore has to be a profile of the Features profile of core?

[KathiSchleidt]

@jerstlouis I'd be VERY interested in your "very concrete example of this, including a full live demonstration" as I again fear we're talking across each other :(

To what extent is your approach related to EDR, so just point at the space you want data for and get it? (whereby I'm currently at a loss as to WHAT EDR returns :( )

It's very hard for me to project what we're doing with STA into the OGC API world as I've yet to get a straight answer on if a collection can contain objects without a geometry. While @cportele had confirmed that non-spatial objects can comprise a collection, this seems to have been rolled back on #149

• If Collections can contain non-spatial objects, the OAPI doc Hylke has provided for STA would be a valid sketch of where we think we're going
• If Collections can only contain spatial objects, I'm currently at a loss of how to provide the full breadth of O&M type stuff (based on real world use cases) can be provided via OAPI

[cportele]

@KathiSchleidt

149 restores that capability. Where Collections.adoc currently states

Geospatial data is the set of all Spatial Things and Temporal Things. Any dataset whose members are restricted to Spatial Things and Temporal Things can be an OAPI Collection.

the PR changes this to

Geospatial datasets include Spatial Things and/or Temporal Things as data items (but not all data items in the dataset have to have a spatial or temporal extent). Any subset of a dataset can be an OGC API Collection.

So, there may be entire collections without a data item with a spatial geometry - as long as there is some collection in the dataset that has a spatial extent (otherwise, why would OGC care?).

[hylkevds]

@rob-metalinkage: I suppose the best example of such a thing would be if one takes the SensorThings API, and changes the urls to be more like OGC API. We're a looooong way away from an OGC API - O&M, especially with O&M being in flux right now, so it's way to soon to start talking conformance classes.

I think the SensorThings API is also a great example of the fact that to get a nice, easy to use, powerful API one can not see those three issues as separate things. They strongly influence each other:

• Things have 0 to many Datastreams that have 0 to many Observations that can all be in-lined. Here your set schema intrudes on your entity schema: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Things?$top=5&$expand=Datastreams
• All entity sets (including in-lined ones) are subject to paging and thus can have nextlinks. Here the service behaviour influences the entity schema and the set schema, since nextlinks for a nested set can not be put at the outer set: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Things?$top=5&$expand=Datastreams($top=2;$expand=Observations($top=2))
• All entity representations can be trimmed down by selecting the properties to return, also for nested entities/sets. Here the the service behaviour influences the entity schema, and the other way around: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Things?$top=5&$select=name&$expand=Datastreams($select=name;$top=2;$expand=Observations($select=result,phenomenonTime;$top=2))
• Filtering can be done over relations, arbitrarily deep. Here the data model influences the service: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Things?$top=5&$filter=Datastreams/ObservedProperty/name%20eq%20%27NO2%27

These are all simple rules that could be applied to any service that has a data model behind it. So yes, we're "conflating" these issues, because they have to be to get a good API...

[dblodgett-usgs]

Per discussion in the OAB just now -- I want to bump this. It would be really nice to have some progress on how a SensorThings API (technically, I guess this means OData) endpoint(s) might be realized as a way to access data within an OGC API Collection.

@cmheazel -- you put this as future work. Has there been any substantive chatter about this since 2020 that you are aware of?

[jerstlouis]

@dblodgett-usgs In terms of recent development, the Connected Systems and Moving Features API (itself an extension to OGC API - Features) may be related to this discussion.

[joanma747]

In my opinion the "datastream" concept in STA was created to "group" observations. Actually it contains the observations of the same sensor, the same thing and the same observedProperty. I believe it can be assimilated to the current concept of "collection".

So, to me it will be possible to do: http://server.bob/STA/collections/{datastreamId}/observations/{observationId} -->returns the observation with the feature of interest expanded http://server.bob/STA/collections/{datastreamId}/sensor --> returns the sensor of this data stream http://server.bob/STA/collections/{datastreamId}/thing --> returns the thing of this data stream including its location and historical locations expanded http://server.bob/STA/collections/{datastreamId}/observedProperty --> returns the description of the observed property

All the query characteristics of ODATA will be unavailable in this endpoint but, at least, we will have the dataset available as collections. And it is easy to make a OpenAPI description of this approach. I'm not talking about replacing the ODATA protocol but only complement it with an OpenAPI approach. I believe it is pointless to expose other objects endpoints; it will make the API unnecessarily complicated.

[hylkevds]

If the goal is to have a STA endpoint merged into an existing OGC-Api endpoint it becomes a matter of:

1. defining what the semantics are of the .../v1.1/ being where it is
2. letting the client know that what is below that .../v1.1/ is SensorThings.

The first is very domain specific. I think that in many cases it is sufficient to have the v1.1/ next to the collections/ path, and have each collection item that is also represented in STA have a link directly to the STA Thing/FoI/Location/Datastream/Sensor that it represents. From there the client can navigate/filter further.

For the second point several options are possible. I'm not completely up-to-date on the OGC-Api spec, so I may have missed some:

1. The STA conformance classes can be added to the /conformance document.
2. A (partial) STA OpenAPI definition (like this one) can be merged into the /api document. That will at least let the client know which basic requests are possible, and what the schema of the returned data is. The fact that the paths look a bit different should not matter, that's what OpenAPI was made for. Of course, if the v1.1/ is merged into many places in the path tree this would really bloat the OpenAPI definition.
3. Label links that point into the STA paths as being of type SensorThings. I don't know what the conventions are for that, but it would let the client explicitly know it can switch to STA. Otherwise the client would have to infer that from the OpenAPI definition.

I think that would be enough for an OGC-Api client to also navigate into the SensorThings API parts, and for a combined OGC-API & STA client to figure out which features can be used where.

[joanma747]

@hylkevds, It really depends on what you want to do here. Your approach is right, but it does not respond to the "verbal" question that I got from the OAB. It was my fault not stating the question I was trying to answer first. Sorry. The question was:

Does the STA represent a dataset? It the answer is "yes", can we map the content to "collections"? This is a fundamental question that goes back to: What is a OGC API?. In my interpretation, the question is not if we can or cannot express STA as a OpenAPI document but if we can map STA to the concept of "dataset" and "collection". One of the fundamental things we are trying to do in OGC API is that the same dataset and the same collections can be represented as features (collections/{collectionId}/items) coverages (collections/{collectionId}/coverage) etc. My proposal above is allowing for a new "view" of the same dataset as observations (collections/{collectionId}/observations) an return a list of STA observations (with their FoI).

Forget about the /v1.1; the advantage of this solution is that the same endpoint could work with ".../collections" as a OGC API and as ".../datastreams()" (and all the other objects) as STA. I'm still not sure if we can have a merged landing page or 2 separated landing pages but this is just a detail as the same "server" could offer 2 landing pages if necessary: server.org/ogcapi/collections and server.org/sta/v1.1/datastreams()...

[rob-metalinkage]

I guess a key concern here is whether it is possible to convey to a client that a particular datastream represents a particular collection - or perhaps that they both represent different parts of a more comprehensive domain model that specifies how they relate. would be good to have a worked example of how this might work, then we can build a building block that includes examples and validators for this pattern.

[cportele]

@joanma747

Does the STA represent a dataset? It the answer is "yes", can we map the content to "collections"? This is a fundamental question that goes back to: What is a OGC API?. In my interpretation, the question is not if we can or cannot express STA as a OpenAPI document but if we can map STA to the concept of "dataset" and "collection". One of the fundamental things we are trying to do in OGC API is that the same dataset and the same collections can be represented as features (collections/{collectionId}/items) coverages (collections/{collectionId}/coverage) etc. My proposal above is allowing for a new "view" of the same dataset as observations (collections/{collectionId}/observations) an return a list of STA observations (with their FoI).

Why invent collections/{collectionId}/observations? I would return the observations at collections/{collectionId}/items. An observation is a feature.

[hylkevds]

@joanma747 indeed, there are two questions (and I think they are often mixed):

• Can one mix STA (or even plain OData) with other OGC-APIs in a way that lets them both keep their identity (and advantages/disadvantages) yet integrates them so a client can cross-walk between the two.
• Can one, in a meaningful way, serve a relational data model (of which the STA data model is just an example) using OGC APIs.

I think @dblodgett-usgs is after the first.

[jerstlouis]

@cportele In the context of the observations on a Feature of Interest, wouldn't observations rather correspond to properties? (measured / observed properties, aka range fields in a coverage).

The idea of a new STA view / access mechanism on the collection does makes sense if it present things in a data model that does not fit simply as a different representation of the Features /items, otherwise if doable mapping it to a feature collection and /items makes perfect sense.

@hylkevds Regarding the relational data model, I believe that is the purpose of OGC API - Joins.

Regarding offering a collection as both STA and OGC API, that makes complete sense to me. Adding a link to an STA end-point to an OGC API collection as an additional access mechanism (with a distinctive link relation type), and vice versa adding a link to the OGC API landing page, collection, and/or items to an STA end-point, would do the trick.

And the use case of an OGC API facade on top of an STA end-point I believe is also another related key use case (the facade could of course include the link to the source STA access mechanism in the collection).

[hylkevds]

If I understand OGC API - Joins correctly, from a client perspective it does in no way deal with relational data models. The source data may be relational, but for the client the result is just a flat collection without any relational information. Like creating an SQL View that still has all the foreign keys missing.

But do correct me if I missed something in the joins spec.

[pzaborowski]

IMO datastreams in STA are great/required as they limit bandwidth - stop- and as side effect limits flexibility/inconsistency points in data. Turning that into a collection make sense if we're able to select data streams per property, time etc when querying. It may be also straightforward in implementations. If not - it may be a problem on the hubs with multiple sources. A simple case is an entity collecting data from bunch of calibrated measurement points with multiple properties, and data in various locations. Does 1-1 mapping to a collection mean that multiple collections per point per property? Would more complicated collections based on the ObservedProperty+normalisation like a unit of measurement, be easier from upstream?

[jerstlouis]

@hylkevds My understanding, based more on assumptions on what I would expect a join API to be than from reading the draft spec, was that the ability for a client to specify what data to join and how implied a relational data model being exposed. I may be mistaken about this being the intent.

[cmheazel]

@dblodgett-usgs Yes Dave, there has been a lot of work in sensor integration, moving features, and ultimately Connected Systems. The Connected Systems SWG will have their kickoff meeting in Frascati to start the transition of our Testbed work into OGC Standards. We have two existing implementations. These implementations act as bridges between non-web sensor systems and a proposed Web API flavor of SWE. Most of these sensor systems are on tactical networks; low bandwidth, high error rate, often disconnected. The data models look nothing like the OGC models. But the translations of the protocols, syntax, and semantics works. Furthermore, these systems work with moving features, moving sensors, motion imagery, and have begun looking at non-terrestrial deployments.

[cmheazel]

In a related development, the 3D+ thread of Testbed 18 has shown that we are not limited to three dimensions. ISO 19107 allows us to define a geometry in Minkowski space-time. An "event" in spacetime is a 4D point which specifies both the spatial and the temporal location in a single coordinate. A volume in spacetime encloses both the spatial and temporal extents. Time becomes just another spatial axis. There will be additional work on this topic in Testbed 19.

[dblodgett-usgs]

Let me clarify. As a producer of geospatial data services that include both sensor things and features, I want a way to expose multiple views of my datasets that are coherently arranged as views of my datasets. I find the OpenAPI / OData dichotomy to be a cumbersome and really just a technical detail that's confusing our client software developers. It would be awfully nice to accommodate both under the same top-level API landing page.

[rob-metalinkage]

reviewing the thread again is interesting :-) I still think the issue is the difference between containers and specialisations, and for different amount of semantics captured in the container model (feature, coverage, OData, Network or any other reusable pattern - we end up with different burdens on the domain model specialisation to capture the dimensions of the data. a single solution can be achieved by conflating these into a single API design, but the reality is that multiple solutions exist for good reasons, and we will need to separate the concerns through describing the relationship between a common conceptual model and these different, but equally valuable, implementation meta-models.

We haven't had a canonical mechanism to describe these relationships - IMHO a legacy of UML and a tendency to build logical models and call them conceptual models.

An activity to formalise a way to describe how different encoding meta-models map to an underlying data model could include machine-readable formalisms as well as validation. It is to this end we are exploring the using of JSON-LD contexts to "semantically uplift" different JSON schemas to an RDF form, which we can then apply validations, transformations and alignment models to.

The challenge is to establish a vehicle whereby we can pool knowledge and effort rather than keep repeating specific viewpoints in the many places these issues keep cropping up. IMHO the OAB could define a set of model-agnostic requirements for APIs to support data models, and then apply STA to this as a test case, rather than argue around specific aspects of STA. The Network and moving features use cases are instructive - perhaps we should collect a set of functional Use Cases which we think require data models to be mapped to multiple API patterns.

[joanma747]

@cportele,

Why invent collections/{collectionId}/observations? I would return the observations at collections/{collectionId}/items. That could be a possibility. Can I negotiate a response format that conforms to the STA entity model (so a JSON encoding that is not GeoJSON but STA-JSON)?.

Another possibility will be that collections/{collectionId}/items/{itemID} respond in a GeoJSON that is a flat representation of the STA entity model: The FeatureOfInterest is the geometry, and the observation result, the dates, the sensor description, the observeProperty description, the thing, the uom and all the rest.... become properties.

[dblodgett-usgs]

@rob-metalinkage

IMHO the OAB could define a set of model-agnostic requirements for APIs to support data models, and then apply STA to this as a test case, rather than argue around specific aspects of STA.

This is exactly what I'm trying to do by calling attention to this issue. I'm not really interested in how observations, sensors, things, features, etc. relate (I mean, I am, but it's not really my focus here). I'm interested in having a coherent OGC landing page for a dataset where we can list all the ways of accessing that dataset.

[rob-metalinkage]

How do we reconcile the "Web Flavor of SWE" with the OGC API suite, (+Building Blocks), the OGC-API-EDR and discussions around STA being an OGC API flavou? can we define a set of OGC API Building Blocks that cover the ISO 19156 model and have expressions in the different data models of the different APIS? does a SWE API perhaps need Geopose (which seems close to the SWE Sensor Model) as well - or need to be aligned to it ?

FYI currently looking at how to create machine readable OGC API building block definitions - and scope of SOSA ontology update to match O&M v3, and how to create reusable building blocks for this and GeoPose

[joanma747]

I'm continuing my particular "thread" here. It is possible to have a single landing page with OGC API and STA since STA has only 2 elements in the landing page: "value" and "serverSettings" and they not interfere with tha OGC API ones.

So an hybrid landing page will look like (OGC API first and STA later)

{
  "title" : "Observation Service",
  "description" : "This is an invented service supporting both OGC API features (with observations) and STA.",
  "links" : [ {
    "rel" : "self",
    "type" : "application/json",
    "title" : "This document",
    "href" : "https://services.bob/STA/v1.1?f=json"
  }, {
    "rel" : "service-desc",
    "type" : "application/vnd.oai.openapi+json;version=3.0",
    "title" : "Formal definition of the API in OpenAPI 3.0",
    "href" : "https://services.bob/STA/v1.1/api?f=json"
  }, {
    "rel" : "conformance",
    "title" : "OGC API conformance classes implemented by this server",
    "href" : "https://services.bob/STA/v1.1/conformance"
  }, {
    "rel" : "data",
    "title" : "Access the data",
    "href" : "https://services.bob/STA/v1.1/collections"
  }
  ],
  "value": [
    {
      "name": "Datastreams",
      "url": "https://cos4cloud.demo.secure-dimensions.de/staplus/v1.1/Datastreams"
    },
    {
      "name": "Observations",
      "url": "https://cos4cloud.demo.secure-dimensions.de/staplus/v1.1/Observations"
    },
    ...
  ],
  "serverSettings": {
    "conformance": [
      "http://www.opengis.net/spec/iot_sensing/1.1/req/datamodel",
      "http://www.opengis.net/spec/iot_sensing/1.1/req/multi-datastream",
      "http://www.opengis.net/spec/iot_sensing/1.1/req/request-data",
    ]
  }
}

Now we only need to agree on what OGC API features could return in this case. Possibilities:

• A GeoJSON feature with observations and they properties as a "flat structure".
• A GeoJSON with STA entities as properties
• A JSON as in STA when you request observations
• ...

[ghobona]

@joanma747 Thanks for sharing the prototype landing page.

If STA offered a landing page like the one shown above, and also offered a /conformance resource, and an API definition document based on OpenAPI, then that would address the Core requirements of OGC API - Common - Part 1: Core .This would place it at the same level of support for the OGC API approach as OGC API - Processes - Part 1: Core.

Note that OGC API - Processes - Part 1: Core does not support OGC API - Common - Part 2: Geospatial Data. In other words, OGC API - Processes - Part 1: Core does not offer a /collections resource. So let's try to integrate a /collections resource into STA, but if that is not possible, then an alternative is for STA to follow the OGC API - Processes - Part 1: Core approach of only supporting OGC API - Common - Part 1: Core.

[rob-metalinkage]

Hi - been looking at the various specifications and trying to work out in practice how this will work. this may be clearer to others, but AFAICT there is provision to reference a JSON schema in the link section for both items (features) and collections (FG-JSON proposes this using a "describedBy" link relation) - and the "type" property linked to a "semantic description". This seems like a workable start - what we need to do then is link the json-schema to the semantics somehow - i.e. to let people know the observation relates to an OM_observation (presumably via sosa:Observation which provides a canonical URI and a semantic model using a canonical model form.) ?It doesnt seem that there is a way to link JSON schemas with JSON-LD contexts (or any other way to do this) that works to annotate schemas directly - only to annotate instance documents.

The other piece that seems to be missing is a way to link a parameter in a query to the schema - again AFAICT this is a "black box" the server is responsible for. Its not obvious how to express that such a parameter is the same as a schema element - to meet the general model of filtering against a schema. If these are known and solved issues, they probably need to be promoted to a more obvious place in the documentation.

Finally - is there any clarity regarding how to define a "building block" so that it can be integrated automatically into an OAS description - how can, for example, the STA model and required query parameters be defined so that they can be somehow imported into an API, without writing a new document for each possible case? The OAS extensions mechanism at https://swagger.io/docs/specification/openapi-extensions/ doesnt provide any mechanism to plugin reusable definitions - only what they should look like in-situ. I feel we should take the basic architecture discussions offline somewhere else, but not sure where or if they are being addressed.

[joanma747]

@rob-metalinkage, IMHO most of your diagnostic is correct.

There is a "$ref" mechanisms that can link to external documents that contains "shared" definitions. In the OGC API Tiles schemas @jerstlouis did a careful implementation of them that goes even further than the one in OGC API tiles (see https://schemas.opengis.net/ogcapi/tiles/part1/1.0/openapi/). I wonder if this is what you are looking for in your last paragraph.

[dblodgett-usgs]

Perhaps some inspiration? https://docs.oasis-open.org/odata/odata-openapi/v1.0/odata-openapi-v1.0.html

[jerstlouis]

@rob-metalinkage More details about the approach mentioned by @joanma747 to define OpenAPI building blocks in such a way that they can easily be tailored and assembled for a particular API definition based on supported capabilities are described in #302 .

The same approach is now also used in OGC API - Tiles, Coverages, Maps, DGGS, Coverages and Processes.

A top-level .yaml file simply includes one file for each supported API path, then swagger-cli bundle can be used to generate a bundled API definition.

[KathiSchleidt]

Sorry for jumping in late here, been a crazy week! Cool thread!

On @cportele statement:

Why invent collections/{collectionId}/observations? I would return the observations at collections/{collectionId}/items. An observation is a feature.

As OMS Observations (core model under STA) are geared to provide information on existing features, where would these be exposed? Separate Collection? How would we provide a link from these features to Observations thereon?

@rob-metalinkage on

"An activity to formalise a way to describe how different encoding meta-models map to an underlying data model"

YES!!! Finding a way to indicate how the conceptual models created within the OGC align with implementations would be wonderful!!! Such a mechanism would be valuable far beyond the OMS/STA context being discussed here. Rob, do you think that this could be included in your slot in the Conceptual Modelling Group on Report on Linked Data and OGC Registries Prototype next week?

Note: mapping to SOSA could cause issues, as not quite aligned. Staying within the OGC context, maybe better to get the modspec bits on OMS running? From there we could then provide suitable links to SOSA, where not quite aligned indicating this.

On the more general request I see here for a simplified or flattened access to OMS-Type data, we've been discussing this issue in the O&M SWG for a while now. To our view, it would be valuable to, in addition to STA, provide a linked simple access endpoint, whereby these simple and complex versions should be aware of each other, provide links. We're still chewing on possible representations for such a simple geometry plus a number approach. Thoughts go towards re-using the ObservableProperty name as an attribute within the properties of a GeoJSON object, in addition, define a convention for the provision of the UoM. For a STA Thing with associated ObservedProperty names "ozone" and "no2", this would lead to properties as follows:

"properties": {
    "phenomenonTime": "2018-01-01T00:00:00Z/2018-01-01T01:00:00Z",
    "ozone": 23,
    "ozone_uom": "ppb",
    "no2": 45,
    "no2_uom": "µg/m3"
}

I'm still not sure where the links would go in an OAPI-F or EDR approach, links to the corresponding datastreams should be provided either within the properties, alternatively in the links section of the API.

I've been working on defining the transformation utilized here with colleagues working under the CODATA DDI-CDI work, as nobody I asked could provide a correct term for this type of transmogrification (shifting the content of the ObservableProperty name to an attribute of the simplified data model). The paper will be published in IASSIST Quarterly this March, here the last draft of Viewpoints on Datapoints

Finally, on merging STA within a wider OGC-API, I do pity the poor developer confronted with an API that drastically modifies it's request structure for one specific branch of the wider API endpoints.

[rob-metalinkage]

Thanks all for the pointers - we are working (this weekend :-) ) to develop a PR for an better building block description template with machine readable components to implement a specific subtype of Observation as a GeoJSON (and FG-JSON) feature, predicated on using interacting building blocks for Observation, a specialised form of Observation that defines the result schema, and a result schema using a third building block (GeoPose).

This stuff will require much testing and consensus building in upcoming testbeds, but we think we can demonstrate how we can support the publication process with tooling that makes a better practice more visible and feasible.

[rob-metalinkage]

PS - there will be a proposal for a SWG in the Agriculture DWG to work on a domain model for agriculture (AIM) that will include in its scope exploration of OGC API implementation - given that its based on SOSA, PROVm GeoSPARQL etc this activity will provide ample opportunity to test the ideas of reusable building blocks. As @KathiSchleidt points out however the commonality of semantics across different representational views is the issue here. There is a proposal from the FG-JSON space to reference both the "semantic type" and the schema ("describedBy") - so we could bridge this gap this way - but thats where SOSA vs OMS becomes a thing - SOSA is something we can reference - and Observation has a stable URI... OMS (and currently most ISO things and all other UML artefacts) are opaque to machine readability and cannot be referenced in a way that adds any direct value for interoperabilty. So for our experiment we'll use SOSA and assume that SOSA V2 will retain backwards compatibility and support OMS semantics with canonical identifiers.