Determine implications of OGC API - Common - Part 2

[ghobona]

The SWG today (2020-05-27) discussed whether the development of OGC API - Common - Part 2: Collections might have implications for OGC API - Coverages. The SWG agreed to revise the document, if possible, to be consistent with OGC API - Common - Part 2: Collections.

[jerstlouis]

it does not fulfil the requirement of completeness.

This is why I suggest to always have the CIS JSON mandatory in {collectionID} (with the option of having the domainset part linked rather than embedded). Regardless of how CIS model-complete the media type is, you are already covered (pun intended) by the CIS JSON provided at /{collectionID}.

[pomakis]

Okay, so in summary, the converging consensus seems to be that a server that supports only GeoTIFF coverages would provide the domainset of the coverage at the {collectionId} endpoint, and the GeoTIFF itself would be served by the {collectionId}/coverage endpoint (accepting optional subset, etc., parameters). There'd be no need for any other coverage-related endpoints.

If this is the way things are going, then I'm all for it. I've been trying to wrap my head around how to implement a GeoTIFF-only coverage service following the current draft of the "OGC API - Coverages - Part 1: Core" document, and it's been making me dizzy.

[jerstlouis]

@pomakis That is part of the simplification that my proposal implies. It was also making me dizzy, and I think things should be as simple as they can to facilitate adoption, while ensuring that we don't lose capabilities to address the use cases originally intended.

[pebau]

glad to sense that the story is converging gradually!

One thought: why specifically the domain set at {collectionId} ? I know I am on thin ice now after my discussion of rangeset, but several components might make sense to be extractable:

• envelope ("where roughly is this coverage in space and time, in my dear WGS84?")
• domainset ("where exactly, in its native CRS?") -- note that, although normally it will be comparatively small, this can become quite large, in the extreme case larger than the range set for very complex coverages
• metadata ("what coverages do we have from IGN-F?"...thinking of INSPIRE here)
• rangetype ("what timeseries do you have?")

Note that I am not (!) opting for getting all the fuss back, it is more about not making domainset stand out.

[jerstlouis]

@pebau My suggestion was to mandate that all of the following always be included at {collectionID}:

• link to {collectionID}/coverage ("rel" : "ogc-coverage")
• envelope (TBD whether Common Part 2 spatiotemporal "extent" already addresses this)
• rangetype (encoded per CIS JSON specs)
• domainset (encoded per CIS JSON specs) either as a property, or as a link to {collectionID}/coverage/domainset (for those use cases when it can be large)

And optionally:

• optional link to coverage metadata
• optional link to {collectionID}/coverage/rangeset, if a naked value set is supported by the media type (e.g. for CIS JSON)

[pebau]

@jerstlouis OK, so who would decide whether a link or the real thing gets delivered? Are you maybe thinking of allowing a ?fmt=... for domainset?

[jerstlouis]

@pebau the server has both options -- "domainSet" property, or ("rel" : "ogc-domainSet"?) link. The recommendation would be that a separate resource and a link should be used if the DomainSet is more complex than simple dimension(s) bounds, and if a linked resource is used, it may support partial queries of the domain set.

At minimum, a CIS JSON DomainSet must be provided (either as a property, or as a link). Links to DomainSet resources in additional formats can always be provided (e.g. even if a CIS JSON property is used) with a different media type ("type" : "mediatype") -- it doesn't matter whether the linked URL is using "?fmt=" or "?f=" or ".something".

[jerstlouis]

Regarding "envelope", I have not brought this up yet because it may be contentious, but

"where roughly is this coverage in space and time, in my dear WGS84?"

is exactly the purpose of the Common - Part 2 "extent" property, with both a spatial and temporal aspect, and already included in {collectionID}, and I would suggest that if it is lacking in any way, it be addressed in the context of that Common property.

I would have expected that any extra information regarding that where & when (or other dimensions) that the coverage world may need, would be found in the DomainSet?

Although I would not call it a deal-breaker for this proposal if we need to keep a redundant "envelope" :)

[dblodgett-usgs]

Scanning through this issue -- very happy to see the progress.

Just want folks to be aware of this issue: https://github.com/opengeospatial/oapi_common/issues/76 and that it is going to the NA to determine how we are going to handle link-relations not to be registered with IANA.

[pebau]

@jerstlouis: ok, if the envelope can be expressed via an extent (can it be n-D?) then why not declare that in OAPI the coverage envelope is given by an extent item.

[pebau]

@jerstlouis:

the server has both options -- "domainSet" property, or ("rel" : "ogc-domainSet"?) link.

Can you show an example of what the corresponding client code is looking like that gets a domainset?

[jerstlouis]

@pebau Currently, /{collectionID}'s extent can be 4-D (CRS84h which is CRS84 + height) + temporal. I don't believe it currently supports more than that, but I assume that additional dimensions could be added (the extent has a "spatial" and "temporal" member, so we could probably add something like an "extra" member?) Also, I may be mistaken, but aren't the bounds of those extra dimensions defined in the DomainSet anyways? If so, having those 4D might be good enough for the quick check of "extent", and the details are in the CIS JSON encoding of the DomainSet.

Re: client code for retrieving domainset, I don't have such client code yet, but it would:

1. Look for a "domainSet" property as part of the ../{collectionID} resource. If it's there, the value follows the CIS JSON encoding
2. If it's not there, it must be part of the links (same links where the "ogc-coverage" link to {collectionID}/coverage is found), and it can find such a link with "rel" : "ogc-domainset" and "type" : "application/json", which will be the CIS JSON encoded DomainSet. This may also mean that this linked domainset resource supports queries (specified in /conformance as a conformance class indicating DomainSet subsetting is supported by the service?)

[pebau]

@jerstlouis re extent: hm, why trying to tweak (there will be issues indeed) and not simply add an n-D bbox to Common? Would be a nice package together with n-D subsetting that was decided to go into Common as well.

[pebau]

@jerstlouis I seem to remember that OAPI should be simple, but now it adds substantial complexity burden on the client programmer's side. Instead of a single GET request now some 20 lines of non-trivial code just to get a domainset?

[jerstlouis]

@pebau re: extent -- it is already the 4D BBOX of Common. I suggest tweaking it (e.g. adding way to specify extra dimensions) to turn it into an n-D extent. This way the part that clients understand will be compatible (they'll at least understand the 4D part).

If we add a separate 'envelope' property, we would still need to include the extent. So that would be somewhat redundant. (plus the DomainSet may also duplicate some of that information?)

Re: OGC API being simple, the property and link patterns, and "follow the link", are something clients must implement anyway for every resource, so it is trivial.

Advantages of not always making the DomainSet a resource when it is simple are that:

• Clients avoid having to request an additional resource and make an extra round-trip to the server
• Servers have one less resources to provide, which is particularly convenient e.g. for the simple case of serving a gridded coverage like a GeoTIFF.

[pebau]

...sorry, all that makes me prefer the so much simpler DescribeCoverage and GetCoverage.

[jerstlouis]

@pebau well, WCS is not going away ;)

Would you prefer if the DomainSet was always a separate resource & link? This would mean always a minimum of 3 resources rather than 2, and always forcing an extra round-trip to the server, even if the domainset is very simple.

.../{collectionID}, .../{collectionID}/coverage, .../{collectionID}/coverage/domainset

RangeType would still always be in {collectionID}. This would not be my preference, but if folks prefer this we could go with that...

What the separated out domainset (as a link) allows vs. DescribeCoverage is that now you can describe most of the coverage aspects, without forcing you to download a potentially very large domainset, and have the option to subset it. So this offers some nice worthwhile flexibility I would think.

[pebau]

@jerstlouis no, I am not in love with GetCoverage (and for sure not with DescribeCoverage), it is just a practical vehicle. And as it represents the state of the art it is my reference for all new ways: does an alternative provide more simplicity or add more complexity?

Currently the complexity gap discussed here is at 200% - 300% if I understand right.

[pebau]

...and why actually do we keep shifting functionality up and down paths? For the machine, it's just strings that are longer or shorter, so a nobrainer. For humans, though, it decides about usability, coding speed and error probability.

My KISS favourite still is: .../coverage/ -- full coverage .../coverage/X -- component, where domainset, rangeset, rangetype, metadata make sense (and if need be, rangeset)

and then if we wish we can lateron add convenience branches like .../coverage/comainset/crs .../coverage/rangetype/red

if a client fears the result of say domainset can be too large use .../coverage/comainset?f=url to obtain a reference.

[ghobona]

Given the following clarification from @jerstlouis at https://github.com/opengeospatial/oapi_common/issues/140#issuecomment-642299075, the WCS SWG needs to decide whether OGC API - Coverages- Part 1: Core has a dependency on OGC API - Common - Part 2: Collections or not.

"As for freedom to specify collections as they see fit, as clarified previously no standard would be required to be based on OGC API Common - Part 2, even though I believe it should be a recommendation of Common for APIs whose purpose is to distribute geospatial data."

If there is no dependency, then I will ask the SWG to pass a Motion at its next meeting to close this issue. If there is a dependency, then the potential implication is that OGC API - Coverages- Part 1: Core would have to wait for OGC API - Common - Part 2: Collections to be completed.

[pebau]

IMHO the Collections concept of OAPI is not adequate to reflect the structure of a coverage. It is relevant, though, to model collections of coverages (which has been stated earlier by several stakeholders). Therefore, there should not be a normative dependency. Rather, coverages should be modelled after the emerging ISO 19123-1 which will also become OGC Abstract Topic 6.

[jerstlouis]

@ghobona Based on the current draft specifications of coverage (including updates agreed upon based on this issue at the last SWG meeting) and the consensus forming in Common, I am of the opinion that there is a dependency on Part 2: Geospatial data.

However this should be a reason to prioritize & accelerate OGC API Common - Part 2, rather than delay OGC API - Coverage.

Regarding @pebau 's concern on collections of coverages, I believe this can be addressed as part of https://github.com/opengeospatial/oapi_common/issues/11 , and this could be an extension to Common - Part 2, and not an immediate requirement to make progress. Any coverage end-points defined for a single coverage will also work tacked onto a coverage part of a collection of coverages.

OGC API - Coverage can depend on OGC API - Common Part 2 while also being modeled after ISO 19123-1. All that Common Part 2 states is that OGC API - Coverage distributes a "collection of geospatial data" as a coverage.

[jerstlouis]

July 15, 2020 SWG:

• Both rangeType and domainSet could be either embedded or referenced at /collections/{collectionID} using a Common-defined mechanism ("$ref" ? "rangeType": {"$ref": "https://....."} -- https://tools.ietf.org/html/draft-pbryan-zyp-json-ref-03 )
• rangeSet can be an optional resource at {collectionID}/coverage/rangeSet , only provided for representations supporting a naked rangeSet -- a link relation type must be defined
• "envelope" will always be included in /collections/{collectionID} resource
• CIS (coverage) metadata for the coverage can optionally be available via a link relation type to be defined, e.g. at /{collectionID}/coverage/metadata

[tomkralidis]

Associated issue as discussed in OGC API - Common https://github.com/opengeospatial/oapi_common/issues/155

[pomakis]

Sorry I missed this 2020-07-15 teleconference; I didn't notice its date and time until it was too late.

I've got a question about the "envelope" decision. Is this "envelope" in addition to the "extent" property that already exists at that endpoint? If so, that seems awfully redundant.

I also want to mention that CubeWerx has a test implementation of 'OGC API - Coverages" up at "https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa". This OGC API service provides a coverage called "Daraa_mosaic_2019" (in addition to a number of vector feature sets). If anybody toys with this and sees any implementation detail that doesn't seem right to them, please let me know.

[tomkralidis]

@pomakis great! Are there associated JSON reprensentations that we can review?

[pomakis]

The only representation we currently support for the coverages themselves is GeoTIFF. However, the domain set and range type can be requested separately via collections/{collectionId}/coverage/domainset and collections/{collectionId}/coverage/rangetype respectively. Each of the endpoints has a few optional query parameters, which are listed in the OpenAPI description at "https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/api".

[jerstlouis]

@pomakis Regarding envelope, I have made the same remark since the beginning of the issue, and we discussed this again today in the call. One reason that envelope persists is because Coverages may define additional dimensions not yet defined in the bounding box. I suggest that bounding box should be extended instead (that should be straightforward as it already has "spatial" and "temporal" and one could add extra dimensions). @Schpidi brought up another concern is CRS other than WGS84, but again isn't the envelope in fact always WGS84 when it concerns the spatial dimensions? ( https://github.com/opengeospatial/ogc_api_coverages/issues/64#issuecomment-638991260 )

I would say that part of the envelope story is not yet entirely settled, and we can open up a further issue to discuss merging it within the extent property (I would be strongly in favor of that). But at least today the decision is that envelope will not be its own resource and will for now be a Coverage extension added to collections/{collectionID}.

Thanks for sharing your implementation. With a quick look, based on the decisions today:

{"href":"https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_mosaic_2019/coverage?f=json","rel":"coverage","type":"application/json","title":"the coverage of this collection"}

This is linked and currently returns an error. If this were to return something proper, based on previous decisions this should be CIS JSON (CoverageJSON would need to use a more specific media type).

{"href":"https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_mosaic_2019/coverage/domainset?f=json","rel":"coverage","type":"application/json","title":"the domain set of the coverage of this collection"},
{"href":"https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_mosaic_2019/coverage/rangetype?f=json","rel":"coverage","type":"application/json","title":"the range type of the coverage of this collection"}

Today we agreed that either linked or embedded for both rangeType and domainSet should be supported. The discussion though leaned towards using the "$href" mechanism, so I am guessing this would mean always using a property rather than using an element of links OR a property. There are on-going discussion about this regarding guidance from Common though (See https://github.com/opengeospatial/oapi_common/issues/155). I would actually prefer using either a links element or a property, but when using links a proper relation type would have to be defined. You are using "coverage" for everything (coverage, rangeSet, domainSet) in your API and I think that is problematic as one cannot distinguish only by "type" (e.g. application/json is defined as the media type for CIS JSON which can represent all of these).

As a side note I would suggest to also add the Daraa DTED in there as well as an example of an elevation coverage ;) (http://maps.ecere.com/geoapi/collections/vtp/Daraa2/Daraa_DTED -- I am planning to update our service as well before the next Coverage code sprint in August).

[nmtoken]

brought up another concern is CRS other than WGS84, but again isn't the envelope in fact always WGS84 when it concerns the spatial dimensions.

Bounding box in a capabilities response has traditionally been a hint of the east west north south area covered by the data set. There has never been a requirement that the data set actually be available in WGS 84 long lat.

because Coverages may define additional dimensions not yet defined in the bounding box.

A coverage may express all of it's dimensions outside of the WGS 84 dimensions specified in the bounding box, not just additional ones. I mean the actual data rather than the hint of the location.

[jerstlouis]

@nmtoken And that would all be perfectly fine with the .../collections/{collectionID} extent property. If for some reason a coverage really does not map any "spatial" dimension at all, then "spatial" could be omitted from the extent. (Though really, if it is real-world data, it normally would be associated with at least one point in space where the measurement occurred?)

[chris-little]

@jerstlouis Two examples:

1. How about sampling the genome of a specific human to construct their full DNA sequence (3.2GB) viewed as a coverage. Even time, within a few thousand years, might not be relevant? ;-)
2. Use wavelength/frequency as a dimension, for averaged radiative properties of the whole earth. Other dimension could be time. In API EDR we assume extents always 4D. If not fully specified, defaults could apply, including error condition, like 'latest', 'top', 'nearest', etc.

[pebau]

Does anybody else think that we are doing a Procrustes exercise here, just to force the more powerful concept of multi-dimensional coverages into feature flatland? ;-)

[KathiSchleidt]

Don't want to add confusion, but we've had some interesting discussions on modelling coverages with the spatial aspect being provided via references to administrative units, in this case one coverage axis refers to these external polygons On the one had this is an edge case, on the other shows how even spatial may be there but in a non-spatial axis

[jerstlouis]

@chris-little , all... Well OK. But we can still include all these non-traditional use cases in a {collectionId} "extent" and omit the "spatial" property. We can either regroup these in some Coverage-specific property for everything that is not temporal and not spatial, or define more specific ones. So as @pomakis suggested again, I strongly support merging envelope with extent, because it really is the exact same thing.

[pomakis]

@jerstlouis wrote:

Thanks for sharing your implementation. With a quick look, based on the decisions today:

{"href":"https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_mosaic_2019/coverage?f=json","rel":"coverage","type":"application/json","title":"the https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_mosaic_2019/coverage?f=json%22,%22rel%22:%22coverage%22,%22type%22:%22application/json%22,%22title%22:%22the coverage of this collection"}

This is linked and currently returns an error. If this were to return something proper, based on previous decisions this should be CIS JSON (CoverageJSON would need to use a more specific media type).

Ooops, that link should be ".geotiff" with a media type of "image/geo+tiff". I'll put up a fix for this within the next couple of days. Of course, without subset, scaleFactor and/or responseHandler parameters, the requested resource would be too large, as it would request the entire coverage at full resolution.

@jerstlouis wrote:

Today we agreed that either embedded or embedded for both rangeType and domainSet should be supported. The discussion though leaned towards using the "$href" mechanism, so I am guessing this would mean always using a property rather than using an element of |links| OR a property. There are on-going discussion about this about guidance from Common though (See opengeospatial/oapi_common#155 https://github.com/opengeospatial/oapi_common/issues/155). I would actually prefer using either a |links| element or a property, but when using links a proper relation type would have to be defined. You are using "coverage" for everything (coverage, rangeSet, domainSet) in your API and I think that is problematic as one cannot distinguish only by "type" (e.g. application/json is defined as the media type for CIS JSON which can represent all of these).

I'm okay with going either way on this. I agree that using links with rel="coverage" for everything is problematic; I implemented it that way for now just to get something going.

@jerstlouis wrote:

As a side note I would suggest to also add the Daraa DTED in there as well as an example of an elevation coverage ;) (http://maps.ecere.com/geoapi/collections/vtp/Daraa2/Daraa_DTED -- I am planning to update our service as well before the next Coverage code sprint in August).

Sure. Where can I get the source data for this coverage?

[jerstlouis]

@pomakis I believe that DTED is at https://portal.ogc.org/index.php?m=projects&a=view&project_id=621&tab=2&artifact_id=92004 for VTP2.

And https://portal.ogc.org/index.php?m=projects&a=view&project_id=550&tab=2&artifact_id=83535 for Testbed 15.

@jeffharrison Could you please confirm it's okay for that DTED data to be shared from demo implementations?

[pvretano]

All, the extent object in common/features is meant to be extensible so other keys can be added that represent the extent that you wish to convey. The extent schema in common/features even says ..."Extensions may add additional members to represent other extents, for example, thermal or pressure ranges."

So let say you have an N-dimensional extent that you wish to communicate ... simply add a new key (e.g. "my_nd_extent"), defined as required and plop it into the extent object. If the your extent object also includes enough information to populate the bbox and temporal keys then copy the relevant information into those keys as well to play nice with clients that happen to know how to handle bbox and temporal but may not recognize your domain/community-specific extent key. { "bbox": { ... }, "temporal": { ... }, "my_nd_extent": {...}, . . . } I see very little reason to define a new extent object when the existing one is extensible! Sorry to jump in like this; I have not been following too closely so if what I say is foolish, please ignore! ;)

[pomakis]

Okay, I've added the Daraa_DTED coverage to the test implementation at "https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa". I've also corrected the rel="coverage" links.

[jerstlouis]

@pomakis Nice! Thanks! :)

[pomakis]

BTW, is anybody aware of a media type for DTED (even an ad-hoc one)? I'd like to allow clients to be able to POST new DTED sources to the https://test.cubewerx.com/cubewerx/cubeserv/demo/ogcapi/Daraa/collections/Daraa_DTED endpoint, but would need it to recognize a DTED-specific Content-Type for this to work. As it stands now, a client would either have to convert the DTED to a single-band integer GeoTIFF first, or package the DTED in a ZIP file.

[ayoumans]

@pomakis I kicked the bear (NGA) because they are the owner/maintainer of the DTED standard, and so far, they don't know of any.

[pebau]

hm, we have a baseline that CIS 1.1 is to be used. Hence, the envelope has to be there, no way around.

If you want to add extent or something else, that's ok (coverage metadata can hold that) with a description how it is derived from domainset or envelope, but the coverage structure must remain...well, the coverage structure. Otherwise it's not a coverage any longer. That's what all the implementations out there that rely on the CIS standard will tell.

[jerstlouis]

@pebau The envelope is there, but would be encoded in the "extent" property. It follows the CIS logical model.

[pebau]

@jerstlouis ok, so we are defining a new coverage encoding? No problem with that, we just maybe should state it clearly and define the concrete mapping, as the CIS format encodings do - they give nice, easy to follow precedence.

[jerstlouis]

@pebau in a sense, yes, but specifically to describe the envelope as the "extent" property of the .../collections/{collectionId} resource. The full CIS JSON encoding of the coverage, including the envelope, can still be made available at .../collections/{collectionId}/coverage .

[pebau]

@jerstlouis so the suggestion is to define two independent data structures, one for a complete coverage response and one for the components? My naive idea was that the component requests simply return coverage components, as their name says. For me that variety is harder to understand and remember.

[jerstlouis]

@pebau What we agreed at the meeting last Wednesday was:

.../{collectionId}/coverage -- returns the whole coverage with all its components supported by the selected representation (e.g. CIS JSON, NetCDF, GeoTIFF... ) .../{collectionId}/coverage/rangeSet -- An optional resource to return only the naked range set, when that is supported for a given representation (linked from .../{collectionId} with a specific relation type). .../{collectionId} -- includes the envelope, rangeType and domainSet. For the envelope I believe we are now coming to an agreement that it should simply be encoded in the "extent" property. The rangeType and domainSet may either be embedded or linked. The "$ref" approach was suggested as one mechanism to achieve this. An alternative would be to use the links together with a specific a relation type for each of the rangeType and domainSet. Suggested paths where those would be linked to would be .../{collectionId}/coverage/rangeType and .../{collectionId}/coverage/domainSet.

[pebau]

@jerstlouis

.../{collectionId} -- includes the envelope, rangeType and domainSet. For the envelope I believe we are now coming to an agreement that it should simply be encoded in the "extent" property.

I disagree strongly. First, a long history of discussion ( in OAPI-Coverages and outside) has shown that it will not be able to reflect the information properly. Second, I see no specification of that. In order to even conduct a meaningful discussion we would need a concrete mapping proposal: how is each envelope item mapped to a corresponding extent item?

Currently we are at the level of hand waving, without a clear discussion basis.

So, how would the supporters suggest to map, concretely?

[pebau]

...generally speaking, for all stated as agreed, can you please point us all to the specification, examples, etc? thanks!

[pebau]

Well OK. But we can still include all these non-traditional use cases in a {collectionId} "extent" and omit the "spatial" property. We can either regroup these in some Coverage-specific property for everything that is not temporal and not spatial, or define more specific ones.

how?

So as @pomakis suggested again, I strongly support merging envelope with extent, because it really is the exact same thing.

merge = enhance extent with envelope information and make it the new OAPI-Common extent?