Subscribe to record changes

[m-mohr]

Have there been considerations about an API that allows subscribing to events, e.g. when a new record has been added? This could include restricting the subscription to certain filters such as time and space, but could in principle be anything that you can express with CQL.

Obviously, OGC API - Records is using HTTP and as such has a focus on pulling information and not pushing them, but it still seems to be an obvious use case that arises from a catalog service.

I've opened it in records as it seems to be the OGC API that fits best, but in theory, this could be developed in any of the OGC WGs, e.g. Features, Commons, or even outside of OGC APIs, e.g. in OpenSearch. So if this is out of scope of Records, is there anything like that in the OGC world that you are aware of?

[m-mohr]

Could be related to https://github.com/opengeospatial/ogcapi-common/issues/231 and OGC PubSub.

[mhogeweg]

Perhaps as part of this discussion it is worth considering the role of off-the-shelf automation engines like integromat, zapier, and a slew of others commonly used to integrate systems using messaging, webhooks or other existing mechanisms

[pvretano]

Actually, there might be something we could do in core and that is include a conformance class about callbacks (see: https://swagger.io/docs/specification/callbacks/). This needs to be discussed in the SWG so I am moving the issue from "Extensions" to "To Do".

[cportele]

To me this is not Core, this is an extension. Core should be restricted to the capabilities that almost every implementation will want to support.

[tomkralidis]

SWG 2021-06-17: make this a generic extension (many standards, many levels), i.e. OGC API - Common. Start writing around requirement with Common in mind and evolve it to Common.

[pvretano]

07-APR-2023: @tomkralidis is writing a conformance class for PubSub for EDR (see: https://github.com/opengeospatial/MetOceanDWG/issues/10). @pvretano will write something similar for records. This will not be part of core.

[tomkralidis]

Note that OGC API - Pub/Sub conformance class is put forth in EDR in https://github.com/opengeospatial/ogcapi-environmental-data-retrieval/pull/426

[rob-metalinkage]

It feels as if pub/sub should be a separate extension that can be implemented optionally.

It should itself have an optional conformance class for querying and exposing the same change metadata that a subscriber would receive in pub/sub, but by poll

[tomkralidis]

It feels as if pub/sub should be a separate extension that can be implemented optionally.

Agree. The OGC API - Pub/Sub conformance class is being put forth initially in EDR but will be implemented in a generic way so that it can be implemented as a building block. In this sense, an OGC API - Records Extension on record changes can have a dependency on OGC API - Pub/Sub and add any applicable bits accordingly.

It should itself have an optional conformance class for querying and exposing the same change metadata that a subscriber would receive in pub/sub, but by poll

Wouldn't polling OGC API - Records (either via a queryable /collections endpoint or .../items) with CQL against the updated queryable help in this regard?

[tomkralidis]

Notes:

• EDR will put forth Part 2: Publish-Subscribe
• an OGC Pub/Sub SWG is being formed (see https://github.com/opengeospatial/pubsub/blob/main/charter/PubSub_SWG_Charter.adoc)

In any case, it makes sense to align Pub/Sub workflows in Records to the above over time.