add link property for service resources

tomkralidis commented 5 years ago

(Not sure if this should be file here or OAPI Common)

We should consider adding a property to the link object.

https://github.com/opengeospatial/WFS_FES/blob/master/core/openapi/schemas/link.yaml

...called name (or something)? The driving use case here is encoding links of, say, WMS endpoints with layer names to be able to bind to a given WMS URL with layer name without have to do a GetCapabilities round trip.

Example:

    {
      "rel": "self",
      "title": "My WMS",
      "name": "my-roads-layer",
      "href": "https://example.org/wms"
    }

cportele commented 5 years ago

How would name be defined? It has to be a general concept that is applicable to all kinds of links and distinct from the other link properties in order to be included in the standard link object, which is basically an implementation of RFC 8288.

tomkralidis commented 5 years ago

Another thought: can we reuse id as defined in https://github.com/opengeospatial/ogcapi-features/blob/master/core/openapi/schemas/collection.yaml#L6-L9 ?

tomkralidis commented 4 years ago

Example in DCAT: https://www.w3.org/TR/vocab-dcat/#a-dataset-available-from-a-service

dr-shorthair commented 4 years ago

and more here: https://www.w3.org/TR/vocab-dcat/#examples-data-service

tomkralidis commented 4 years ago

Example based on https://github.com/opengeospatial/ogcapi-records/issues/13#issuecomment-529902484:

    {
      "rel": "self",
      "title": "My WMS",
      "id": "my-roads-layer",
      "href": "https://example.org/wms"
    }

dr-shorthair commented 4 years ago

JSON-LD and TTL encoded examples are here https://github.com/w3c/dxwg/tree/gh-pages/dcat/examples

pvretano commented 4 years ago

@tomkralidis @dr-shorthair If service A offers/operates on/manipulates layer B then the catalog record for A should have a link in its links section to the catalogue record for layer B and visa-versa. I don't see what adding the id key to the link object (thus breaking compatible with the current encoding standard for links) buys us ... sorry it is late here and my head is fuzzy from a long day so please walk me through it!

tomkralidis commented 4 years ago

@pvretano the goal here is to be able to operate on a resource with a known API pattern that binds based on the metadata record. Imagine one discovers a dataset metadata record which has a link to a WMS 1.3.0 which serves the data via WMS.

How does one express this link? If we provide like:

    {
      "rel": "self",
      "title": "My WMS",
      "href": "https://example.org/wms"
    }

...without knowing the layer name in which the WMS relates to the data, one would have to do a full GetCapabilities and peruse the given layer list/tree to find the layer in question.

Adding the WMS layer name (or WFS FeatureType/WCS Coverage/etc.) to the link model would allow the client to construct, say, a GetMap/GetFeature/GetCoverage request using the minimum binding information in the link model as well as their application context (bbox, width, height). This is exactly the workflow for CSW search/discovery in QGIS.

Of course, with the evolving OGC API standards this becomes less of an issue (given the resource name is now part of the endpoint), but the OGC WxS suite of standards will be around for quite some time.

pvretano commented 4 years ago

@tomkralidis I presume that if you found the metadata record for a "dataset", it would include links to all the metadata records for the "layers" that are part of that "dataset" and those "layer" records would include links to the WMS that serves them. Of course, the "dataset" record could also include a link to the WMS. This is sounding more like an issue of what one chooses to make the "discoverable" resources and how those resources are linked. Of course, I am basing this description on the pattern established with the CSW. When the CSW "harvested" metadata for an OGC service, it also harvested metadata about each layer/feature type/coverage that the service offered such that binding to the service was fairly straight forward without the need for another round trip to get the capabilities document.

cportele commented 4 years ago

@tomkralidis - If understand your point correctly, then what you describe is not a link. It is a new convention with parameters that requires knowledge about OGC Web Services. In my view, we should keep this outside of OGC API and separate from the link object.

I do not fully understand the context, but maybe this could be represented as a URI template (RFC 6570)? For example, something like https://example.org/wms/service=WMS&version=1.3.0&request=GetCapabilities&layer={layer}&bbox={minlong},{minlat},{maxlong},{maxlat}&....

URI templates aren't links either, but they are pretty common, part of the Web architecture and not an OGC-specific convention. We already use URI templates in the Tiles API.

tomkralidis commented 4 years ago

Thanks for the feedback @pvretano / @cportele . Consider the following dataset record:

...and in particular the following snippet:

<dct:references scheme="OGC:WMS">https://geo.woudc.org/ows</dct:references>

At this point, one has to do a GetCapabilities and peruse to find the associated WMS layer. If the above snippet were represented like (for example):

<dct:references scheme="OGC:WMS" name="totalozone">https://geo.woudc.org/ows</dct:references>

A client would know enough to detect 1./ It's a WMS and 2./ the related layer name. At this point, the client could add the given WMS layer to it's viewport and construct a WMS GetMap URL accordingly (yes, things like CRS and such are another matter). Note this is also discussed in more detail at https://github.com/geopython/pycsw/wiki/Geonode-notes

@cportele I really like the URI template approach. Is there an example of how this is used in OGC API - Tiles?

mhogeweg commented 4 years ago

There's implied knowledge in this design. does OGC:WMS cover all versions of WMS? display scales? and is the layer name the best way to identify a layer or would the layer identifier be preferred? what if you want to reference the entire web MAP service, not an individual layer? is the goal to circumvent GetCapabilities? isn't the point here to define what a record for a resource looks like in a catalog, regardless of the various potential uses of that resource?

dr-shorthair commented 4 years ago

When looking at this issue in the context of DCAT v2 we used

dcterms:conformsTo to indicate the 'standard' that the service (or dataset) conforms to (i.e. the 'scheme')
dcat:endpointDescription to link to the specific endpoint details (i.e. the 'capabilities')

This was strongly influenced by the W*S patterns.

mhogeweg commented 4 years ago

The idea of a scheme attribute is fine. Over the years we have had to develop various libraries to parse urls to distinguish different resource types when harvesting metadata. The schemes themselves never have seem to be well defined, so that would be something for the spec to address also.

tomkralidis commented 4 years ago

@mhogeweg :

does OGC:WMS cover all versions of WMS?

Version negotiation?

display scales?

Agree, there is a sweet spot on when to circumvent a GetCapabilities round trip

and is the layer name the best way to identify a layer or would the layer identifier be preferred?

Layer name (i.e. Layer/Name to feed LAYERS={layer-name}

what if you want to reference the entire web MAP service, not an individual layer?

One could either provide a record of the entire service, or in a dataset record a link to the service with no additional OGC:WMS -ish properties

is the goal to circumvent GetCapabilities? isn't the point here to define what a record for a resource looks like in a catalog, regardless of the various potential uses of that resource?

The above articulates the "search catalogue and add layer to map" paradigm. Note that we (OSGeo community) had also created https://github.com/OSGeo/Cat-Interop in an attempt to define a set of "link types" to be able to detect resource types from link "objects".

mhogeweg commented 4 years ago

@tomkralidis from that repo, this list is the key: https://github.com/OSGeo/Cat-Interop/blob/master/LinkPropertyLookupTable.csv. That list should be maintained by OGC and as far as Esri service types is concerned, the list can be extended quite a bit. See for example: https://enterprise.arcgis.com/en/server/latest/publish-services/linux/what-types-of-services-can-you-publish.htm.

tomkralidis commented 4 years ago

Thanks for the info @mhogeweg, and totally agree on managing this in OGC and extending it. I would imagine the outcome of this issue can help decide, moving forward, if/where such a list of "link types" is best managed. I do like @cportele 's idea of URI templates. Perhaps the OGC API - Records core record model can have an object (like gmd:CI_OnlineResource?) to implement URI templates.

cportele commented 4 years ago

@tomkralidis

I really like the URI template approach. Is there an example of how this is used in OGC API - Tiles?

For an example, see here: https://services.interactive-instruments.de/t15/daraa/collections/TransportationGroundCrv/tiles?f=json

This has the following templated link:

{
    "rel" : "item",
    "type" : "application/vnd.mapbox-vector-tile",
    "title" : "Mapbox vector tiles; the link is a URI template where {tileMatrix}/{tileRow}/{tileCol} is the tile based on the tiling scheme {tileMatrixSetId}",
    "href" : "https://services.interactive-instruments.de/t15/daraa/collections/TransportationGroundCrv/tiles/{tileMatrixSetId}/{tileMatrix}/{tileRow}/{tileCol}?f=mvt",
    "templated" : "true"
}

The tileMatrixSetId values are enumerated in the resource and the other three are typically used by a JavaScript mapping API or a tile harvester to access the desired tiles in the tile matrix set.

See https://github.com/opengeospatial/OGC-API-Tiles/blob/master/standard/clause_7_tile_core.adoc#response-2.

tomkralidis commented 4 years ago

@cportele this would work IMO. If there's no objection we can move forward in this direction (which looks like adding templated to link.yaml. We should also document this in the spec.

cportele commented 4 years ago

By the way, the approach reuses the approach to URI templates by HAL:

5.2. templated

The "templated" property is OPTIONAL.

Its value is boolean and SHOULD be true when the Link Object's "href" property is a URI Template.

Its value SHOULD be considered false if it is undefined or any other value than true.

pvretano commented 4 years ago

@cportele @tomkralidis Again, I must be missing something. @tomkralidis mentioned earlier that the issue is the following: (1) client searches the catalogue and finds a dataset record (2) dataset record includes a link to a WMS but without knowing the layer name, client must do a GetCapabilities round trip to get the list of valid layer names. How does using a templated URL help this situation? Don't you still need know that value to replace for the substitution variable (i.e. layer name)? If so, then you still need to do a GetCapabilities round trip to get the layer names. So, what am I missing here?

My thinking was that the catalogue would be a linked (i.e. associated) set of records that provide (among other things) the information necessary to bind to a service. Finding any linked bit of information (dataset, layer, service, styles, etc.) will lead you to other linked (i.e. associated) records so that you can gather the information you need to bind to a service.

B.T.W. This https://github.com/OSGeo/Cat-Interop/LinkPropertyLookupTable.csv is interesting! It looks an awful lot like this: http://docs.opengeospatial.org/per/16-043.html#_standard_identifiers.

tomkralidis commented 4 years ago

Using a templated link in this case would have the layer name hardcoded because it is part of the metadata record per se. The only templated items would be what's in the context of the given application (bbox, width/height/etc.). Still, no GetCapabilities round trip required.

B.T.W. This https://github.com/OSGeo/Cat-Interop/LinkPropertyLookupTable.csv is interesting! It looks an awful lot like this: http://docs.opengeospatial.org/per/16-043.html#_standard_identifiers.

It certainly does! There was much community effort around defining this lookup table (circa 2014) as well as implementation (for example, pycsw and QGIS implement the lookup table) as well numerous projects generating ISO documents.

cportele commented 4 years ago

Meeting 2020-02-24:

We want to support templated links in general. The link relation type will identify the types of templates and their parameters.
The LinkPropertyLookupTable is related and needs to be managed by OGC. Start the work here and discuss moving the management to the OGC-NA. The link relation types should be identified by URIs. The list needs to be kept up-to-date and it should be easy to update it (e.g. by submitting a pull request). @pvretano will open a new issue to move this topic to a separate issue.

tomkralidis commented 4 years ago

Meeting 2020-05-04:

add to document describing templates when adding links to catalogue records

pvretano commented 3 years ago

SWG MEETING 08-FEB-2021: https://github.com/opengeospatial/ogcapi-common/issues/187#issuecomment-775166732 we will leave this open and track common on this issue. "templated" is currently used here and in tiles.

pvretano commented 3 years ago

22-FEB-2021: @tomkralidis already added most of what is required to the specification. Assigning it to @tomkralidis and moving to "to be drafted". @tomkralidis will review to make sure that everything is in the specification and close.

tomkralidis commented 3 years ago

Schema, examples, and document updated/consistent (https://docs.opengeospatial.org/DRAFTS/20-004.html#_associations_and_links). Closing.

pvgenuchten commented 3 years ago

How does templating deal with post requests, some of the standards (WFS, CSW) do not offer full capabilities when called via GET. Well at least you'll be able to extract some of the relevant parameters from the templated GET request and use it to construct a POST request.

There is one caveat with this templated approach. Which is also a big caveat of OWS, a plain endpoint returns a error 500. If a user mistakingly clicks a templated url, the server will return that same error 500

In OGC API we could improve that by indicating the server to return a valid response by filling in defaults or provide an index page with a listing of potential values to replace

opengeospatial / ogcapi-records

add link property for service resources #13