pangeo-forge / roadmap

Pangeo Forge public roadmap
Creative Commons Attribution 4.0 International
19 stars 4 forks source link

Implement OGC EDR API for catalog search #39

Open cisaacstern opened 2 years ago

cisaacstern commented 2 years ago

In a recent Pangeo Showcase, @shanemill1 & @solson-nws presented a compelling implementation of the OGC API - Environment Data Retrieval (EDR):

The presentation highlighted the utility of this API for performing dimensional/faceted search across a heterogenous data catalog. As of opening this issue, we are planning to use the SpatioTemporal Asset Catalog (STAC) and related tooling as our default catalog interface for Pangeo Forge. Some users may be more familiar/comfortable with the OGC API, however, so we should implement this technology as well.

This is not currently a shovel-ready project, as it is likely dependent on a range of other ongoing projects, including:

As such, work on this issue is as much a project management / coordination task is it is an engineering one, at this stage.

TomAugspurger commented 2 years ago

Might want to edit the post to indicate that this is specifically OGC API - Environment Data Retrieval (EDR).

I think that the UK Met Office is also involved in defining the standard. Maybe @nbarlowATI knows who is working on that effort?

We (the Planetary Computer team) will also be looking into how OGC API - EDR complements / overlaps with STAC.

cisaacstern commented 2 years ago

Edited, thanks Tom! Glad to know we'll be able coordinate on this.

ShaneMill1 commented 2 years ago

Hi all,

@m-burgoyne and @chris-little from the UK Met Office are editors of the OGC EDR-API.

Also, @solson-nws, @tomkralidis and myself have been looking heavily into the different search aspects and how OGC API's can assist with this.

I figured I would tag them here in case there is interest in collaboration!

cisaacstern commented 2 years ago

@abkfenris, I started this Issue this week as an aggregation point for resources and discussion on OGC EDR API implementations. You mentioned at today's Pangeo community meeting that you've been working with pygeoapi to implement this API in for the HYCOM ocean model. At the moment, we haven't begun working on an implementation for Pangeo Forge, but if you have links, takeaways, questions, etc. you'd like to share from your experience, we'd love to hear them.

tomkralidis commented 2 years ago

Thanks for the cc @ShaneMill1. As mentioned, we have been working on clarifying search architecture in the OGC MetOcean Domain Working Group), and where search fits at the "discovery" level as well as the "data search" level. For discovery, we at the OGC API - Records Standards Working Group are working on metadata models and best practices using OGC API - Records. Note that this is also being discussed at WMO to enable low barrier/mass market friendly weather/climate/water search.

Having said this, on the technology side, pygeoapi implements OGC API - Records, EDR, STAC and other standards, and is extensible (via data providers/adapters/plugins), so may be worth a look.

abkfenris commented 2 years ago

I really just started playing around with pygeoapi yesterday, so I don't have a whole lot to share yet.

I think I'll probably be diving into the provider plugin system to deal with the weirdness of FVCOM's (what I meant to write and say, but I've also been fighting with HYCOM recently) grid. It looks like I will be able to extend the existing xarray_edr Provider.

chris-little commented 2 years ago

@cisaacstern , @abkfenris Perhaps I should state that EDR API is now an official OGC standard. Hence stable, but also open to comments, RFCs, etc. We (the EDR API Standard Working Group) will be working on examples and guide materials in the next few months. You are welcome to raise issues on the EDR API Github. We have ideas for versions 1.1, 1.2 etc., hence backwards compatible, and no plans for a breaking version 2.0. HTH, Chris