Collection of Collections

[cmheazel]

Can a collection contain other collections? The GFM says yes, but that may be too much complexity to address in simple core.

[cportele]

See also https://github.com/opengeospatial/WFS_FES/issues/90 and https://github.com/opengeospatial/WFS_FES/issues/76

[cportele]

If we are talking about collections of collections we should also make sure we define the distinction between dataset (as in DCAT, see https://www.w3.org/TR/vocab-dcat/#vocabulary-overview) and our feature/coverage/etc collection resources. Datasets have a specific role (see schema.org, the Google Dataset Search, the Data on the Web Best Practices) and we should fit into this picture.

In WFS 3.0 Core we support the simplest case, "every WFS instance provides access to a single dataset" (see e.g. the Scope statement).

In the strawman OGC API resource diagram in which I have added the feature/process/coverage/tile/style resources that I found in drafts I have also included one approach how to support multiple dataset as an input for discussion.

[nmtoken]

For coverages we have collection of coverages, is that a similar concept?

[jerstlouis]

I believe it would be extremely useful to define a proper optional collection of collections conformance class as part of OGC API - Common (Collections).

See also discussions on recursive datasets and large number of collections in https://github.com/opengeospatial/oapi_common/issues/17, https://github.com/opengeospatial/oapi_common/issues/69 and https://github.com/opengeospatial/oapi_common/issues/111 .

[cmheazel]

A comment received from Uwe Voges: I was wondering how the use case will be covered that there may be hierarchies of collections (see also STAC) ? This will include the case that some hierarchy levels may be searchable for items (records) while others may not ... Example:

Meteosat Catalogue (Searchable for all Meteosat products) MTG (Searchable just for MTG products) MSG (Searchable just for MSG) Metop Catalogue (Searchable for all Metop products) EPS (Searchable for EPS products) EPS-SG (Searchable for EPS-SG products) Sentinel Catalogue (not searchable) Sentinel3 Catalogue (Searchable for Sentinel3 products)

Ok, one idea could be to provide different API´s with an own landing page etc for every hierarchyLevel....

Another option could be to have the /collections resource which may not even return links to single collections (/collection) but also links to other hierarchy levels of collections (/collections) (rel="collections" like the rel "data" in the landing page).

I tried ad hoc a JSON encoding of the example above which could be provided by the link from the landing page (rel="data") (just a few metadata elements considered here):

{ "collections": [ { "id": "msgCat", "type": "catalogue", "title": "Meteosat Catalogue", "links": [ { "href": "http://.../meteosatCat/collections", "rel": "collections", "title": "At this endpoint you can retrieve hypermedia controls that allow you to navigate sub-collections, e.g. MTG, MSG. Here you can retrieve descriptions of these sub-catalogues as well as hypermedia controls that allow you to query the sub-catalogues" }, { "href": "http://.../meteosatCat/collections/meteosat", "rel": "collection", "title": "At this endpoint you can retrieve a description of this catalogue as well as hypermedia controls that allow you to query the catalogue." } ] }, { "id": "metopCat", "type": "catalogue", "title": "Metop Catalogue", "links": [ { "href": "http://.../metopCat/collections", "rel": "collections", "title": "At this endpoint you can retrieve hypermedia controls that allow you to navigate sub-collections, e.g. EPS-SG. Here you can retrieve descriptions of these sub-catalogues as well as hypermedia controls that allow you to query the sub-catalogues" }, { "href": "http://.../metopCat/collections/metop", "rel": "collection", "title": "At this endpoint you can retrieve a description of this catalogue as well as hypermedia controls that allow you to query the metop catalogue." } ] }, { "id": "sentinel3Cat", "type": "catalogue", "title": "Sentinel3 Catalogue", "links": [ { "href": "http://.../sentinelCat/collections", "rel": "collections", "title": "At this endpoint you can retrieve hypermedia controls that allow you to navigate sub-collections, e.g. Sentinel3. Here you can retrieve descriptions of these sub-catalogues as well as hypermedia controls that allow you to query the sub-catalogues" } ] } ] }

[cmheazel]

Copied from the Wiki at the request of Dave Blodgett

One of the most contentious issues we face in API development is consensus on standard resource paths. API-Features uses /collections/{collectionId}/items. Other SWGs plan to use /collections/{collectionId}/coverage, /collections/{collectionId}/maps, etc. Which is correct?

There appears to be consensus on /collections/{collectionId}. At least for those resources which are typically distributed as collections. The final part of the path tells you something about the collection. A taxonomy of these path fragments could look something like this:

/coverage -> Measurements organized in a regular or irregular grid

/map -> 2D Rendered information displayable for human visual consumption

/tile -> A subset of a dataset based on area of interest and level of detail

/items -> Any

Wait a minute -- "Any"!!

What use is a path fragment which can be anything?

Actually, it’s the most useful of all.

OGC Web Services are built around a taxonomy of data types; maps, features, coverages, etc. It’s a data provider centric view of the world. But what if we took a user-centric view of APIs. What if collections are built around the information that a user needs to do their job?

Consider a first responder. They are going to need a building model, routes of ingress and egress, photographs, overhead imagery, utilities, an elevation model, and all sorts of other supporting data. Everything they need to respond to an incident at a specific location. And they are going to want it in one neat little package, a collection. So how is our first responder going to use “/collections/110_Maple_Avenue/”? /maps, /coverage, /routes all have value. But how do you access the collection contents as a whole? /items.

/items allows us to locate and access any content in the collection regardless of type. So the fundamental access path for any collection is /collections/{collectionId}/items. This does not mean that the other paths don’t have value, just at a different level.

We can view /coverage, /maps, etc. as profiles of /items. They both restrict and extend /items. They restrict in that they only provide access to a specific type of item in the collection. They extend in that they bring to the user a whole collection of tools which are only useful for that type of resource. So “/collections/110_Maple_Avenue/coverage” brings to the first responder all of the sub-setting and processing tools which come with coverage data. But only for that subset of the collection which is coverage data.

So what does /items actually return? A Feature collection.

Now that the outrage has had a chance to settle,

We tend to think of Features as vector data. But at its most fundamental level, a Feature is an empty container. It has neither type, identity, nor properties. So a Feature can grow up to be just about anything. Which is exactly the sort of construct we are looking for.

Under this approach, API-Common would define the bbox, limit, and datetime parameters and how they would be applied to a generic data item. Other API standards would specialize those capabilities for the unique properties of the resources they address.

[rob-metalinkage]

when a problem is very complex in one level of abstraction - as I feel this one is - it can help to change the domain of discourse to a different level (e.g. fourier transforms between time and frequency domains)

In this case I think the problem is too narrow a focus, and hence different solutions for smuggling additional information into the places you can see..

At a higher level of abstraction I believe you can look at "context" for metadata - and understand there is always an element of "out-of-band" metadata - the question is what is necessary and what is acceptable.

In the "classic" SOA a client may go to a catalog and discover a service end-point and out-of-band metadata about the service type - at which point they use this context to formulate a query (get capabilities) about additional metadata the server will self-report.

The same pattern holds true for data descriptions. To what extent is the data responsible for self-reporting. In general, as little as possible, otherwise you need to find and access and interrogate too much before you can select useful data.

So much of the context comes down to some form of "out-of-band" description - some form of catalog. This can vary from a hyper-local (hardcoded in a client) catalog, to service self-description to aggregated catalogs with well-known service interfaces. Its still pushing the data description problem to some out-of-band metadata construct.

So now the question is really one of minimising impedance when a client accesses such out-of-band sources of context - i.e. the interoperability of the description. This includes a description of how the data model of the data maps onto the data meta-model of the service (in this case collections/items/features)

As @cmheazel suggests - you have all the mechanisms you need - its a choice of how you use them, and the point here IMHO is understanding how to map interoperable metadata forms onto the two different models supported by the service: container based addressing and reporting of metadata (capabilities). We can push all this to the issue of publishing Feature Types with sufficient richness and interoperability that such mappings can be immediately inferred - but its probably worth developing test cases and examples for what ever pattern you choose.

[dblodgett-usgs]

I thought I responded to this but my comment is missing.

@cmheazel : When I called out the wiki post, and said "collection" of collection issues, I thought it was a pun!

I linked to the collection TAG trying to make the suggestion that I thought your wiki post was a good frame for bringing some closure to a lot of similar issues related to using collections. Not just this one. But I'll take it. We need to get this one settled!

[jerstlouis]

The approach currently implemented in our client & service:

• /collections returns a list of a all collections

• /collections/{collectionId} returns the resource for that collection (containing the links to the rel items, tiles etc.)

• As an extension, a special character within the {collectionId} is understood to be a hierarchy delimiter, and the service supports returning a partial list of collection if the client requests /collections/{partial-collectionId}.

This works nicely with multi-layer vector tiles as well, in which case a {partial-collectionId} can be both a collection linking to multi-layer /tiles end-point, as well as list the individual collections.

At the moment that special character happens to be a '/', and the invalidity of this can be debated, but in practice it has caused no interoperability problem with our service during 2+ years of TIEs. This could be changed to use another character and specify that character with a property, e.g. "hierarchySeparator" : "-".

Features also state that an OGC API end-point should contain a single dataset, and that hierarchies of multiple datasets should be implemented entirely at another level (i.e. higher than the landing page).

I propose that combined hierarchies of datasets and collections could be implemented as an integrated optional capability, while still being able to mark which level constitutes a dataset. There could also potentially be landing pages at different levels...

[dblodgett-usgs]

@jerstlouis -- I'm having a hard time connecting the first part of your comment to the last. Maybe some more for instance API paths would help?

Are you saying we might have a hierarchy of OGC-API landing pages?

What API path would dereference to a dataset (landing page) and what API path would dereference to a collection that is subordinate to that dataset?

I think you are saying that an integrated dataset-landing page might be something like:
/collections/demo/ would behave as an integrated collection with endpoints like:
/collections/demo/tiles and /collections/demo/items?

Subordinate to that integrated collection could then be the members at API paths like:
/collections/demo-roads/ which would dereference to a dataset (landing page) for demo-roads?

Is this what you are getting at or am I misunderstanding?

[jerstlouis]

@dblodgett-usgs The first part of my comment we have implemented and used in many TIEs (but currently with slashes rather than dashes). e.g.

• http://maps.ecere.com/geoapi/collections
• http://maps.ecere.com/geoapi/collections/vtp
• http://maps.ecere.com/geoapi/collections/vtp/Daraa2
• http://maps.ecere.com/geoapi/collections/vtp/Daraa2/AgricultureSrf

When requesting the JSON representation of these, the whole list of collections is returned, but the HTML representation only shows the immediate hierarchy level.

The latter part of my comment about the landing page at each level, I guess would imply combining those list of collections with the elements normally found in a landing page (e.g. having an api description, in addition to a list of collections and/or tiles & items)

[dblodgett-usgs]

OK, I think I follow where you are going but some example API paths that illustrate the concept with some functional description would be helpful. Thanks for the live example links, but again, some functional description of what these are would be a bit help.

[jerstlouis]

http://maps.ecere.com/geoapi/collections -- This is the list of all datasets/collections on the server http://maps.ecere.com/geoapi/collections/vtp -- This is a filtered list of datasets/collections under "vtp" http://maps.ecere.com/geoapi/collections/vtp/Daraa2 -- This is a list of collections for the dataset "vtp/Daraa2" (something would need to identify it as this point in the hierarchy being a single dataset) http://maps.ecere.com/geoapi/collections/vtp/Daraa2/AgricultureSrf -- This is the AgricultureSrf collection inside the "vtp/Daraa2" dataset.

I was suggesting we could possibly define such an extension of combined hierarchy of datasets & collections, so that both datasets and collections can have an arbitrary number of hierarchy levels, sharing the same hierarchical mechanism.

Does that help? Not sure what you're asking as "functional description".

[dblodgett-usgs]

Yes it does -- to a point. The flexible typing of end points is really confusing and API path templates would help.

I feel like the root /collections isn't needed at all. If you want an ad-hoc cataloging mechanism, then that's what you need.

Why not drop the collections path literal all together and have a GNOSIS dataset catalog that links to landing pages for each of the datasets you want to mash up? The hierarchy could be documented however meets your use case.

You would end up with a flat collection of API end points (datasets) that could be orchestrated into new API endpoints.

e.g. http://maps.ecere.com/geoapi/vtp (which is a mash up of others) http://maps.ecere.com/geoapi/Daraa2 (which is a mash up of others) http://maps.ecere.com/geoapi/AgricultureSrf (which is just a dataset)

All of which would have various functions available -- {root}/collections for feature collections, {root}/tiles for tiles, {root}/coverages for coverages.

The API path complexity just doesn't help, IMHO.

[jerstlouis]

@dblodgett-usgs I don't see how this provides the desired functionality of being able to browse both datasets and collections in a hierarchical manner?

There is a need for both hiearchies of collections (if a dataset is inherently organized in a hierarchical manner), and of datasets (to categorize very large number of datasets).

This is what I am proposing here (with the possibility of combining both), but what you just suggested is to have a flat list of datasets, and a flat list of collections.

[dblodgett-usgs]

... it doesn't handle the complexity -- that's a feature, not a bug.

We are working on common building blocks not a comprehensive SDI. We need building blocks that allow us to create complexity, not complexity that happens to be composed of building blocks.

If we get a good set of building blocks and start seeing common patterns of complexity, maybe we encode them as best practices or even standard implementation patterns, but IMHO, we need to pursue simple building blocks first and foremost that get us to a baseline rather than wandering around aimlessly pretending we can handle the complexity of these kinds of mash up implementations before we have even shown we can handle the simple use cases.

[jerstlouis]

@dblodgett-usgs

The desire for hierarchical collections, as an extension, is what this issue is about.

I am not sure I understand your point. A lot of simple and more complex building blocks have been shown to work, but we are still side-stepping this important issue.

[dblodgett-usgs]

Hold on... "what is a collection" is not settled yet so hierarchical collections as an extension is debating about uncertainty -- ill-defined concepts and functionality.

What I'm getting at is that there is a path forward here where we can focus on building simple components [that can be assembled into hierarchies] that can be composed into systems.

If, once we've learned how to crawl (i.e. get the components right), and experimented sufficiently with how to run (i.e. build systems and hierarchies), we have some implementation patterns that are worth standardizing, we can and should.

Pretending that we can make proposals for shared integrated practices, without well defined components that are being integrated isn't productive. What I'm trying to do is find productive paths forward to build well-defined, well-separated, simple, building blocks that allow us to get to complexity.

[jerstlouis]

@dblodgett-usgs Well, "what is a collection" is debated in many other separate issues, including #111, #122, this new interesting one #140...

This one was about hierarchy. While I agree that the basics should be settled first, sometimes I like to keep believing that this was settled in Toulouse, as both the published Features standard, and the draft Coverage specifications support a collection base path and providing "rel" : "items" links to vector and/or raster data at:

/collections/{collectionID}

[jeffharrison]

The prototype OGC API for the 3D Pilot is advancing for many Participants, TIEs are underway and demos today for the first time showed heirarchies.

The heirarchies approach being tested in the 3D Pilot largely emulates existing OGC practices with respect to volume hierarchies, and seeks to make it accessible for a wide variety of content types, scales and granularities (as OGC Staff indicated). A participating component can implement TIE functionality or not, but it is part of the hands-on collaborative prototyping process.

In a larger context and to paraphrase OGC Staff, it's fine that OGC API Features decided to go with a flat list of collections to emphasize simplicity where appropriate. But not all OGC API's are, nor are intended to be, extended versions of OGC API Features. Not all building blocks of OGC API Common need to be, or even should be, extensions of OGC API Features.

There is no Common Collections heirarchies guidance that went missing ...it's under development now and I welcome the opportunity to work on it.

Best Regards, Jeff

[jeffharrison]

I understand you may feel the approach recently discussed with respect to 'children' in hierarchies may not be compatible with other approaches, but I think it's OK for a Pilot to test different approaches, assess results and document findings for the community.

If you feel you didn't have input into something, let's bring it up to the project team and get it sorted out. I'm not exactly sure what that is in reference to.

Best Regards, Jeff

[jerstlouis]

@jeffharrison We want a common mechanism for hierarchy of collections. Is there a chance that Features and Coverage would agree to this mechanism for hierarchy of data using their API?

Sure, we can test different approaches in Pilots, but if we are testing functionality which people have been asking for since the beginning and we know should be in Common (as this issue suggests), we need to consider whether there's a possibility for this to work elsewhere.

The collection hierarchy is common functionality desirable across all data types.

[jeffharrison]

Jerome,

I'm not sure if we want a 'common mechanism for a hierarchy of collections'.

You and I have talked quite a bit about collections and the 'need' for it, and I've participated in many projects and read many GitHubs and I have questions... I'm not trying to start an argument, I'm just asking because I don't know the answers... so

Why do we need collections?

Why can't applications link to landing pages and navigate from there?

Best Regards, Jeff

[terryidol]

Jerome,

This discussion is not appropriate for the Open API group.

The other Participants have reached the agreed upon approach and are TIEing successfully.

Per the agreement of the Pilot, the work in the Pilot should not be discussed outside until the Pilot is complete. Right now, you are presenting a one sided view of the Pilot. The other Participants may (would) not agree.

Please move this conversation and all conversations back to the Pilot that concern the Pilot.

On Fri, May 22, 2020 at 3:55 PM Jerome St-Louis notifications@github.com wrote:

@jeffharrison https://github.com/jeffharrison This is in reference to what we are now discussing, the 'children' property in a collection description (/collections/{collectionID}) as a way to implement hierarchies of collection.

We want a common mechanism for hierarchy of collections. Is there are chance that Features and Coverage would agree to this mechanism for hierarchy of data using their API?

Sure, we can test different approaches in Pilots, but if we are testing functionality which people have been asking for since the beginning and we know should be in Common (as this issue suggests), we need to consider whether there's a possibility for this to work elsewhere.

I feel like a lot of the 3D Pilot discussions that led to the 'children' approach result is that the hierarchy of collections, and the spatially-organized hierarchy of data within a single collection (the bounding volume hierarchy) are being conflated.

While the BVH is a 3D-specific concept, the collection hierarchy is common functionality desirable across all data types.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/opengeospatial/oapi_common/issues/11#issuecomment-632888176, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEXDMTV4YAE7SPNK7KL3TFTRS3KDLANCNFSM4G76JEHA .

-- Terry Idol, Ph.D. Innovation Program Open Geospatial Consortium

Office: 703 407 2643

The OGC: Making Location Count... www.opengeospatial.org

-- https://www.ogc.org/webinars

[jerstlouis]

@terryidol Sorry, it was not my intention to discuss any specifics of the Pilot, or present a one-sided view of the pilot, but it seems I have done that. I apologize. I removed all references to the pilot from my comments.

I was just trying to highlight how hierarchies of collections are desired by various groups, and how side-stepping defining a common building block for this is problematic to on-going activities. In the end if we want a common building block for hierarchies of collections, this is the issue where this should be discussed.

@jeffharrison The idea of collections as a generalized concept for geospatial data layer, and a uniform way to link to their metadata such as the bounding box/volume, or attach other modules like Tiles, is very useful. This is being discussed in the many other Common issues tagged Collections.

[jeffharrison]

(Note - some of my recent posts may appear a bit disjointed. They were in response to now deleted comments)

In the spirit of helping OGC APIs and the community advance, let's review some examples of how they work. We do this because discussions seem to focus alot on collections and of course there's more to the picture.

So, how might we get a vector tile that's Not Web Mercator using OGC API - Tiles and linking? From the 'Quick Start' guide...

--> Visit the landing page

• discover a link to collections

--> Visit the /collections page

• look for the list of {collectionId}s and select an id.

--> Visit your favorite collection using /collections/{collectionId}

• look for a link of rel='tiles'

--> Visit the link of rel='tiles'

-- look for the 'links' property and select a 'link' with rel='item' (templated) and 'type' with the format you like

-- look for the 'tileMatrixSetLink' property to know about the TileMatrixSetId values

--> if the TileMatrixSetId is unknown

• look for the 'tileMatrixSetURI' in the 'tileMatrixSetLink' property to figure out the values and limits on {tileMatrix} {tileRow} {tileCol}.

--> if the 'tileMatrixSetLimits' property exists in the 'tileMatrixSetLink' tune the values of {tileMatrix} {tileRow} {tileCol} accordingly

--> Use the URL template and substitute the variables by values to get a URL

• get your vector tile!

Best Regards, Jeff

[jerstlouis]

@jeffharrison Good morning Jeff.

That's all right and good. So there are collections involved there in that process. And the value of Collections is that you could retrieve geospatial data using that exact same method or very similar method, even if your data is not vector tiles e.g.:

• Retrieving vector data in other way (e.g. as whole Features)
• Retrieving coverages also this way (tiled or not)
• Retrieving 3D data in the same way (tiled or not)

It also allows to directly link to a Collection, e.g. to specify the input of a process, or for your GIS tool to remember the data in your map project, or context document.

This particular issue here now is about wishing to present these collections in a hierarchical manner. For example, the Natural Earth public domain dataset (e.g. http://maps.ecere.com/geoapi/collections/NaturalEarth) has feature collections organized as either representing cultural or physical features, and e.g. the bathymetry (physical features) is further split per depths. My recent comment was also remarking how you may also want to organize datasets in such a hierarchy (e.g. organized by projects, or provider, or themes...). Sometimes you want the hierarchy within a dataset, sometimes you want a hierarchy outside the dataset, sometimes both. So this is why I suggested considering maybe functionality which could achieve both at the same time, e.g. by 'marking' what consitutes the dataset level. While it is true that one could implement this hierarchy entirely separately from the OGC API (that is slightly easier for the datasets hierarchy), I (and many others, possibly you as well @jeffharrison, though I admit I am a bit confused as to what your position on this is) feel like such hierarchies are important and should represent a well-defined standardized module of the OGC API, and they make sense regardless of data types. This is what this issue is about, and hopefully once this is resolved, this common hierarchical functionality could be used consistently across the different data types (3D or not) to get to the data (as a whole).

All of this has nothing to do with bounding volume hierarchies inside the data of a single collection, e.g. as used in 3D Tiles and i3s. In OGC API - Common (Collections) speak, as currenly drafted and used by Features and Coverages, this issue is about organizing in a hierarchical manner those collections beofre, and in order to, eventually reach a "rel" : "items", using only Common building blocks, which will be offered in one or more representations. Then from there there can still be data hierarchy, e.g. BVH, or TileMatrixSet (also hierarchical, though the hierarchy is fixed), or Discrete Global Grid Systems, defined by a separate module.

[jeffharrison]

This discussion is more fundamental than 3D hierarchies.

One key discussion point is that using the term collections is not mandatory.

So in an OGC API - Tiles you could have...

GET /{geospatialResource}/tiles/{tileMatrixSetId}/{tileMatrixId}/{tileRow}/{tileCol}

...which returns a tile - a representation of real-world elements at a given resolution restricted by the selected Tile Matrix Set.

And the identifier of the {geospatialResource} could be "/collections/{collectionId}" or a coverage or another geospatialResource. Correct?

Best Regards, Jeff

[jerstlouis]

@jeffharrison this issue is about hierarchy of what I like to call geospatial data layers, which I currently refer to as "collections", which at least up to recently was the agreed upon terminology. Both Features and the current Coverage draft use /collections/{collectionID}. For coverages, the {collectionID} is the {coverageID}.

As to whether what goes in that {geospatialResource} in your URLs is fixed to /collections/{collectionId} or not, that was the case in the latest OGC API - Common (Collections) draft. I think it still is at the moment, but there are now suggestions that it should not be. I am extremely concerned by that.

I would very much like it to stay this way, because I am advocating to handle different types of data in a more similar manner for the common aspects. I would like "getting to the data" working the same way, regardless of what type of data it is.

If your coverages are in one place, and your 3D data another, and your vector data in another, the "common approach to geospatial data" is already starting to drift apart. Your landing page with a "rel" : "data" link points to a single place (i.e. currently to /collections), so if your dataset is made up of vector, coverage and 3D data, that's not going to work to have the geospatial resources in seperate places.

If we can all get over the name 'collection' which displeases some (myself first and foremost), and agree that the most granular uniquely identifiyable geospatial data resource before starting to partition the data spatially and temporally is called a "collection", I think we will be able to move along with the various OGC API specifications and build interoperable modular APIs.

The other use case that the current approach supports is being able to retrieve the same data (collection) as both coverage & 3D, or vector and 3D, or coverage and vector.

For example this collection here: http://maps.ecere.com/geoapi/collections/CampPendleton/Buildings will be retrievable as both 3D Tiles and vector features.

This elevation data here: http://maps.ecere.com/geoapi/collections/CampPendleton/Elevation will be retrievable as both 3D Tiles and coverage.

[jeffharrison]

Nothing I've shown today is 'mine'. The examples are from the OGC API - Tiles 'Quick Guide'.

A key point is OGC lately spends a lot of time discussing the term 'collections', but it's not mandatory to use it. The identifier of a {geospatialResource} could be "/collections/{collectionId}" or a coverage or another geospatialResource.

Best Regards, Jeff

[jerstlouis]

@jeffharrison by saying "your URL" I meant the URL written in your comment.

As I wrote in my last comment, coverages currently use also /collections/{collectionID}, where the {collectionID} is equivalent to the {coverageID}.

Any other geospatial resource could do the same, and I believe this is the only possible way that a dataset could contain features and some other types of geospatial data, as your landing page can only have a single "rel" : "data" link which must point to /collections for Features.

Datasets containing different types of geospatial data are common (e.g. Natural Earth, GeoPackages, CDB...).

I do believe that the OGC API - Common recommendation should be to use /collections/{collectionID} for all geospatial data resources, and recent clarifications that this is not mandatory IMHO is a rabbithole (because of those various reasons I mentioned in this and previous comments).

The recent decision to separate the Core (as only /, /api, and /conformance) from Collections so that it can be released on its own while the Collections issues are resolved does not change the fact that we should still try to solve those Collections issues. It should not be seen as a nail in the coffin of Collections or of /collections/{collectionID}.

[jeffharrison]

No one is suggesting 'nails in coffins' ;-) The key point is OGC spends a lot of time discussing the term 'collections', which is good, but it's not mandatory to use it. The identifier of a {geospatialResource} could be "/collections/{collectionId}" or a coverage or another geospatialResource. That's a very reasonable position in my opinion.

Best Regards, Jeff

[jerstlouis]

@jeffharrison please read my comments and you will understand why it's not reasonable.

[jeffharrison]

OGC shouldn't mandate the use of the term 'collections' as the identifier for all geospatial resources. But at this point in the OGC API development process it's reasonable for OGC to say the identifier of a {geospatialResource} could be "/collections/{collectionId}" or a coverage or another geospatialResource.

Best Regards, Jeff

[jerstlouis]

@jeffharrison My opinion is that in fact it should, for the many reasons I detailed in this issue and others.

Especially those 2 reasons:

• To support the ability to offer a dataset containing geospatial resources of different types (e.g. coverage, vector features and 3D data), as is the case in many data sets (NaturalEarth, GeoPackages, CDB), because a landing page can only have a single "rel" : "data"
• To support the ability to offer a particular data layer (i.e. collection) as different types (e.g. 3D Tiles & features, features & coverage, etc.), e.g. each as "rel":"items" but different "type"

And again, coverage currently uses /collections/{collectionID}.

It's okay if you don't see how this matters, or disagree, but I don't think I can explain this in a clearer way without repeating myself even more, so we should probably leave it at that. Enjoy the rest of the weekend :)

All the best,

-Jerome

[jeffharrison]

Also at the risk of repeating myself ;-) I'm good with the language coming out of parts of OGC API - Tiles saying...

The identifier of a {geospatialResource} could be "/collections/{collectionId}" or a coverage or another geospatialResource.

I think it's healthy for OGC API to have some extensibility in its view of geospatial interoperability.

Have a great weekend!

Best Regards, Jeff

[jerstlouis]

@jeffharrison I am also good with that language in OGC API - Tiles, because it could be tagged on to something not implementing Collections. But my opinion for OGC API modules being defined for geospatial data resources is that they should generally implement Collections.

One important thing I forgot to point out is that the value of /collections/{collectionID} also lies in having a common schema for what it returns used by all geospatial data resources API implementing OGC API - Common (Collections). e.g., the presence of "rel" : "items" as a link to the data, "rel" : "tiles" as an attachment point for the Tiles API, spatial and temporal extents, metadata, and so on.

Collections is a common attachment point so that all OGC API modules can work together. Why is this even controversial? I honestly don't understand. I just want to see those OGC API modules truly work together in an interoperable manner.

[jeffharrison]

There have been hundreds of comments on why collections is controversial.

Best Regards, Jeff

[jerstlouis]

@jeffharrison There have been hundreds of comments complaining about collections (many of them coming from you in fact ;) ), but my feeling is that they all stem from a dislike of the name, not an objection to the concept. I dislike the name myself. But the concept of a generic geospatial data layer is present in all GIS software and incredibly useful. I doubt that fundamentally anyone has an actual issue with the concept. It is only the name that irks.

There would have been an opportunity to fix the name before Features was standardized, and I tried my very best to push for that because I knew this would happen and it would stand in the way of a consistent interoperable OGC API. Alas, that did not happen. But there is still a last glimpse of hope for this to work, if we can accept the name 'collection' and move along. It is a small price to pay for the wonderful interoperability that lies ahead.

After all, what's in a name? That which we call a rose by any other name would smell as sweet.

[jeffharrison]

Fundamentally, I think the future lies in resource templating and linking.

Hardcoding just leads to brittle architectures.

Thanks and Regards, Jeff

[jerstlouis]

@jeffharrison OK let's say we drop the literal 'collections' part in /collections/{collectionID}.

I don't really have an issue with that, other than Features currently requiring it. So if one wanted to serve a datasets containing Features + something else, or offer something as Features and something else, then in that case one would have to stick to /collections, unless Features could be relaxed to allow that to be something else, and just following the "rel" : "data" from the landing page. Not standardizing /collections costs one extra round-trip to the server to get to your data -- it's not ideal, but perhaps not a showstopper.

So if we put that aside, it would be fine for a landing page to have "rel" : "data", and follow the link to the list of geospatial data resources, and the content of that response (listing each data resource), as well as the content of < relDataURL >/{dataResourceID} would, return something following what is currently being returned by /collections and /collections/{collectionID}. One would still want to standardize the use of "rel" : "items" as the way to retrieve the actual data itself (e.g. the coverage, or the features, or the 3D Tiles tileset, or the i3s root tile), spatiotemporal extents, etc. This mechanism could be called OGC API - Common (Collections), or it could be called OGC API - Common (Geospatial data) or anything else. The important thing is to have these common schemas and link relations.

[jeffharrison]

Jerome,

Very interesting proposal. I'm curious to hear what others think about a landing page having something like "rel" : "data" as a potential way forward towards an 'OGC API - Common (Geospatial Data) '

Thanks and Regards, Jeff

[dblodgett-usgs]

@jerstlouis -- I think your latest idea is close to but a bit different to what I posed as a starting point for #140 ?

I'm having a tough time following your summary without it being broken down with some example paths and more detailed description of the returned information and link relations.

Would you be willing to flesh the proposal out in #140 with some examples of what it would mean in the context of an existing Features API and how it would work for other APIs (coverages and tiles maybe)?

One other question -- could we have a world where an OGC-API had:

/api
/conformance
/collections (for feature collections of the dataset)
/coverages (for coverages of the dataset)
/tiles (for tiles of the dataset)
/{dataresourceIDs} (for sub-datasets)

Where each {dataresourceID} was its self an OGC-API? The "one rel data per landing page" issue is maybe an issue here? I'm not super versed in the nuances there.

[jerstlouis]

@dblodgett-usgs

The "one rel data per landing page" issue is maybe an issue here?

Yes, that is the issue here.

Also the current OGC API - Coverage draft specification currently uses /collections/{collectionID} just like Features, not /coverages (since Toulouse TC in November 2019).

Your list of resources has me thinking you are trying to map things directly to classic OGC services ;) /coverages -> WCS, /collections -> WFS, /tiles -> WMTS.

My main issue with this is that a specific coverage and a specific feature collection could be offered as tiles. And the same dataset could be offered as both coverage and features.

I find much more natural and harmonious an API which has resources organized as: /api /conformance /roses /roses/{roseID}/coverage (for coverage) /roses/{roseID}/items (for features) /roses/{roseID}/{other resources specific to other types} /roses/{roseID}/tiles -- although my original preference would have been to put that after /coverage or /items, so that you could distinguish between tiled features and tiled coverage of the same data

where roses is whatever it has to be to make everyone happy, or is left to the decision of the service implementer.

If you have a default server-side rendering of that dataset, then:

/map and /map/tiles might also make sense.

As does /roses/{roseID}/map and /roses/{roseID}/map/tiles for a server-side rendering of a particular rose.

The classic OGC services can still be provided alongside these, and either the old services over the OGC API, or the new OGC API over the old services can be implemented as a facade.

I also like that you are proposing /{dataresourceIDs} for sub-datasets here.

I will take another look at #140 as well where this whole discussion would probably have been more appropriate, until we come back to /{dataresourceIDs} for sub-datasets.

[dblodgett-usgs]

Thanks for fleshing this out @jerstlouis.

Re: the "rel: data" issue,

Yes, that is the issue here.

or that is an issue here? Big difference. For a dataset distribution, I find "rel: data" to be close to meaningless and certainly arbitrary. It was invented for the first features spec and is not an IANA link relation. Maybe someone can point us to the relevant discussion of its selection in the issue tracker for Features?

I would find it much more useful to have API Common say that each OGC API should define a unique link relation for its endpoint(s) that form part of the dataset-distribution. Somewhat unfortunately, Features' relation in that would be "data" -- early bird gets the worm I suppose.

Re: the data-resource:/api cardinality issue

Your list of resources has me thinking you are trying to map things directly to classic OGC services ;) /coverages -> WCS, /collections -> WFS, /tiles -> WMTS.

I list those resources because they are functions that operate on the top-level resource, the dataset distribution.

I just went back in time a bit and found @cportele's wisdom over in: https://github.com/opengeospatial/ogcapi-features/issues/76#issuecomment-424252836

I think that discussion is very relevant and remains open with a discussion that kind of ends without follow up, that is unfortunate. The same thread is now spread widely and difficult to follow.

Some commonality can be found iin that /roses/{roseID} is a dataset distribution -- and so is /. What I'm arguing for is that we figure out how to get dataset distributions right independent of the complexity of creating hierarchies of them. A server is free to implement a hierarchical path structure for it's OGC-API dataset distributions and if we get to a place where it's useful to specify some kind of standard behavior around those, then we could do that.

Thank you for taking the time to follow up on #140. I will follow up on this there.

[cportele]

@dblodgett-usgs

1. Yes, "data" in Features is kind of under-specified, because it was never that important, because of the fixed path structure. I will add more on that in #140 when I get to it. Also note that while "data" will be continue to be used for backwards compatibility in Features, for Common we need a fresh start anyhow as we cannot register "data" with IANA. Same with "items".

2. Thanks for digging up the old discussion from Features. Indeed that topic is still open and my comments from that time are still valid. It would be nice, if it would have been resolved by now, but from a Features perspective it was never a priority work item (the priority work items are documented in the SWG charter). It seems that this is different from the perspective of Common or other resource types. @jerstlouis is clearly very keen about this. I will get back to that in #140.

3. With the new version of DCAT the relationship to distributions became more complex and Features Core 1.0.1 will change the language a bit, because it references DCAT 2. DCAT 2 added a constraint that any distribution has a single media type. This was not present in DCAT 1, but because my API may share the same feature as GeoJSON, GML, CSV, HTML, etc. the Collections resource is in most cases no longer a distribution, but has links to multiple distributions /collections/collection/items with the media type as a "parameter".

[jerstlouis]

Recently, we have updated our new experimental OGC API server to use : rather than / within a {collectionId} to indicate an implied hierarchy, which may be recognized by clients that supports this. I suggest this could be a conformance class of OGC API - Common - Part 2: Geospatial Data to indicate that a server uses this approach. This enables Collections of Collections.

The validity of : in the URL path was put in question before, but a discussion on the topic is found here on StackOverflow. In particular, it points out that this is both valid according to the specifications (though it is a bit of a deep dive to understand this in detail), but more simply and conclusively that this is used on Wikipedia (e.g. http://en.wikipedia.org/wiki/Template:Welcome), one of the top 10 websites in the world.

A request for /collections as JSON will still return all collections, making this interoperable with clients not understanding hierarchies, but an HTML representation may instead decide to only list the top-level resources and links to dive in further inside the hierarchy. When requesting a parent collection description resource specifically, we extend the collectionInfo schema by also adding a collections property listing sub-collections as well (e.g. https://maps.ecere.com/ogcapi/collections/Daraa?f=json). The hierarchical collections conformance class could specify and standardize this behavior.

A related discussion is a structure for linking to and listing (and potentially filtering as well) multiple datasets APIs offered from a centralized OGC API service, which e.g. may have processes which can take as input data from any of the datasets offered, as examplified here. A similar : based separators could be used to organize datasets in a hierarchy as well, although perhaps to accommodate a very large number of datasets and because there are no compatibility constraints, it might be better in that case to not always return the full hierarchy at the top level, even for JSON representation.

[joanma747]

An alternative to hierarchy is to use a themes classification as detailed in issue #170

[joanma747]

In practical terms, this is only about reserving ":" for the hierarchy. This means that servers should not use ":" in their names except to indicate hierarchy. Normally, the ":" is commonly used to separate "short namespaces" from "codes" (e.g. ogc:wmts). This is not so different form a hierarchy of "two" levels. There is already some restriction for collection names (are they listed someplace?): they cannot contain a "/" or a "?" (indication of a query part of the url) or a '#' (indication of the start of the "hash" part or the URL. So adding another restriction to this is not so problematic. I suppose all of this symbols could be escaped in the URL; including a ":" that is not a hierarchical separation.

[jerstlouis]

@joanma747 Correct. And this extra requirement could possibly only apply if you declare conformance to a hierarchical collection conformance class.

There is one more aspect to it which is to be able to extend the collection description schema to include a "collections" : [ ... ] property listing all sub-collections when you request a parent. e.g. https://maps.ecere.com/ogcapi/collections/NaturalEarth:physical:bathymetry will include a listing of all sub-collections, in addition to any information about the top-level collection itself (e.g. like a resource to render a map of all bathymetry sub-collections).

When requesting https://maps.ecere.com/ogcapi/collections/?f=json, all collections are always returned for the JSON representation to be compatible with clients not supporting the hierarchy extension.

However the HTML representation can be free to only return the top-level collections e.g. at https://maps.ecere.com/ogcapi/collections.