search

radiantearth / stac-spec

SpatioTemporal Asset Catalog specification - making geospatial assets openly searchable and crawlable
https://stacspec.org
Apache License 2.0
796 stars 178 forks source link

consideration of OpenSearch EO extensions #16

Closed tomkralidis closed 5 years ago

tomkralidis commented 7 years ago

Hi all: wondering if there has been any consideration on the OGC OpenSearch Extension for Earth Observation Implementation Standard? Seems like use cases for collections and products are covered, and leveraging an existing standard could provide the value of broad interoperability as well.

Thoughts/comments? Apologies in advance if I missed this in the docs somewhere.

cholmes commented 7 years ago

Some consideration, though not a ton. I think a few people took a look at it and had trouble getting a real handle on it and how to use it. Your link above is a lot more interesting than what I've found on it - seems like a nice sensible description.

Lots of changes happening at the sprint, and it looks like the core is not going to be EO specific - is going to attempt to be general, to search sar, drones, video, satellites, etc. So core will be mostly space and time, and then extension mechanisms. But the EO 'extension' set would be great to align with OGC OpenSearch.

cc @matthewhanson

uvoges commented 7 years ago

Just to let you know: in the OGC EO-PMOS Standards Working Group we´re currently standardizing a new version of the OpenSearch EO-Extension (13-026r9 -> aligned with OWC Context, CEOS,...). Accompanied by a GeoJSON(-LD) response encoding (OGC 17-047), GeoJSON(-LD) based Metadata Profile for EO-products (granules) (OGC 17-003) and a GeoJSON(-LD) based Metadata Profile for EO-collections (just in draft status). We hope to have final version by 1/2018.

m-mohr commented 6 years ago

@uvoges Where can we get more information about the current state of the new version of the OpenSearch EO-extension incl. the JSON response encoding? Sounds really interesting to me, but couldn't find much information on the web.

uvoges commented 6 years ago

Dear Matthias,

The spec is more or less finished and currently under OGC review (before going to public comment). I'm not the expert in the OGC policies. I guess getting insight in the current status depends on your OGC member status and if you have subscribed to an observer agreement.

Maybe Scott can answer...

Best regards, Uwe Voges

uvoges commented 6 years ago

All,

The draft standard will be available to the public as soon as the next review process (OGC Architecture Board) is complete. That will occur in the next two weeks. In the meantime, Uwe, you can send a draft to Matthias if your SWG agrees.

Best Regards, Scott

On May 9, 2018, at 8:21 AM, Uwe Voges u.voges@conterra.de wrote:

Dear Matthias,

The spec is more or less finished and currently under OGC review (before going to public comment). I'm not the expert in the OGC policies. I guess getting insight in the current status depends on your OGC member status and if you have subscribed to an observer agreement.

Maybe Scott can answer...

Best regards, Uwe Voges

Outlook for Android https://aka.ms/ghei36 herunterladen

Von: Matthias Mohr Gesendet: Mittwoch, 9. Mai 14:57 Betreff: Re: [radiantearth/stac-spec] consideration of OpenSearch EO extensions (#16) An: radiantearth/stac-spec Cc: Uwe Voges, Mention

@uvoges https://github.com/uvoges Where can we get more information about the current state of the new version of the OpenSearch EO-extension incl. the JSON response encoding? Sounds really interesting to me, but couldn't find much information on the web. — You are receiving this because you were mentioned. Reply to this email directly, view https://github.com/radiantearth/stac-spec/issues/16#issuecomment-387729466 it on GitHub https://github.com/radiantearth/stac-spec/issues/16#issuecomment-387729466, or mute https://github.com/notifications/unsubscribe-auth/ALkt55a320Nm2k4KfXaxMcIFBpTDpTv6ks5twuewgaJpZM4QDltYthe https://github.com/notifications/unsubscribe-auth/ALkt55a320Nm2k4KfXaxMcIFBpTDpTv6ks5twuewgaJpZM4QDltY https://github.com/notifications/unsubscribe-auth/ALkt55a320Nm2k4KfXaxMcIFBpTDpTv6ks5twuewgaJpZM4QDltYthread https://github.com/notifications/unsubscribe-auth/ALkt55a320Nm2k4KfXaxMcIFBpTDpTv6ks5twuewgaJpZM4QDltY.

m-mohr commented 6 years ago

@uvoges Uwe, thank you for checking on that. Having it in two weeks or so is soon enough for me. Where will the draft standard be published? Grüße auf die andere Seite Münsters. ;)

uvoges commented 6 years ago

...ah, vom IfGI? Schick Dir eine Version dann in 2/3 Wochen. Kannst mich ja noch mal erinnern...

VG, Uwe

cholmes commented 6 years ago

@m-mohr - did you manage to review this? Would be good to figure out if we can align some...

m-mohr commented 6 years ago

OGC released a draft open for comments on July 4, deadline was August 3. Unfortunately I missed the deadline as July was very busy and holiday season and the spec is hundreds of pages long or so.

I had a brief look. I found it very hard to really get into it due to its massive length. Some comments follow...

Most interesting is probably the search functionality:

Regarding the JSON response format:

Regarding the JSON metadata format:

Preliminary conclusion: Too complex for us and seems not easy to align (naming of search parameters, different JSON encoding regarding properties, ...).

PS: Not sure whether you can access the draft documents. We are an OGC member and seem to have access to more data than the public.

uvoges commented 6 years ago

Dear Matthias,

Before ging into detail, maybe next week, I would like to inform you that this candidate standard is currently already implemented by a lot oft space agency including:

So simple does not always mean sufficient for all usw Cases...

Regards, Uwe

uvoges commented 6 years ago

Dear Matthias,

I guess to develop something new from scratch which is easy to use and smart (being specific, considering a few use cases) is relatively simple and makes often sense. But when more and more organisations become involved the use cases to cover, the interopeability aspects, test cases.... and so the requirements increase. That is a reason why the spec is lengthy. It covers lots of real life EO use cases and provides lots of specifications including detailed searches as required by: automated processings, Atom & JSON & JSON-LD clients, an ontology for EO products... I guess search request and response are both interesting:

The spec is already used by EO Integration platforms as e.g. CEOS/CWIC, ESA/FEDEO, Eumetsat/Wikeo etc and implemented by lots of Space Agencies (See my last email)..NASA and ESA have developed a test environments to test the implementations which will be integrated.

But it maybe also be good to have additional specialized and easy to use APIs for specific usw cases. We do the same in different projects: defining simple and easy to use Rest APIs based on Swagger/OpenAPI. Please note: the latter will be discussed to replace the current OSDD/XML.

Some specific notes:

Regarding the JSON encodings maybe Yves can provide further details.

Hope this clarifies a bit the intention of OpenSearch-EO.

Best regards, Uwe

m-mohr commented 6 years ago

Dear @uvoges,

I really appreciate that you took the time to get into more details. I'm sure OGC OpenSearch EO is well specified and the adoption from several space agencies provides evidence that it can cover a lot of use cases. I never meant to say it is not good, just very hard to get into any maybe not what we are looking for for STAC. In fact, I really like that the JSON encoding got introduced. It is also not generally a bad thing to have a lengthy specification (several hundred pages?!), but it makes it hard for developers to get into it. Especially, smaller companies/projects may struggle with that. STAC for example is mainly run as a side-project by most people, I think. Therefore, nobody has the resources to go through the whole spec and implement it. That said, I'd expect a much shorter version which describes how to get started if you really would like to increase adoption. Is there something like that (planned)? For example, we may consider just implementing the required search parameters and a basic JSON response including all required fields to get basic compatibility, but for me the spec makes it very hard to achieve that.

We may consider adopting or complying to CSW4/CAT4. If they stick to OpenSearch as a conformance level, we may adopt it through that.

Again, highly appreciated that you take the time to explain and contribute here.

Best regards, Matthias

sjskhalsa commented 6 years ago

CEOS has published several versions of an OpenSearch Best Practice Document which may be a more accessible doc for implementing the OS EO Spec

m-mohr commented 5 years ago

I think we decided for now to not add OpenSearch to the STAC core due to its complexity, but it could be added as an additional query extension to the API. See #500 for ideas how to integrate more query languages.

One additional issue (that's purely my personal opinion) is that OpenSearch EO responses (not sure about the exact name, but I saw a presentation on it recently) are not STAC compatible AND not GeoJSON compatible. The specification defines a properties field in FeatureCollections, which is not allowed. This is something that should be fixed first.

uvoges commented 5 years ago

Dear Matthias,

1. OpenSearch is a query capability which is (from an interface point of view) as simple as any other search capability which you support with the dynamic STAC catalogue. It is just a sequence of Name/value pairs provided in a HTTP/GET request...and GeoJSON or Atom response...

2. With the OGC OpenSearch EO Extension we are not aligned with STAC as STAC does currently Not cover the more complex use cases which were defined by ESA, NASA, Eumetsat, Vito, DLR, etc in the Context of CEOS/CWIC, FedEO etc. We got much more requirements for the query capabilities and for the responses than STAC currently supports -> See OGC 13-026r9, 17-047, 17-003

3. To align with the OGC stack, here OWS Context, we needed to accept the GeoJSON Bug which is a Bug in OWS Context, Not in OpenSearch-EO.

Thanks and best regards, Uwe

Holen Sie sich Outlook für Androidhttps://aka.ms/ghei36

From: Matthias Mohr notifications@github.com Sent: Thursday, July 18, 2019 5:27:39 PM To: radiantearth/stac-spec Cc: Uwe Voges - con terra GmbH; Mention Subject: Re: [radiantearth/stac-spec] consideration of OpenSearch EO extensions (#16)

I think we decided for now to not add OpenSearch to the STAC core due to its complexity, but it could be added as an additional query extension to the API. See #500https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fradiantearth%2Fstac-spec%2Fpull%2F500&data=01%7C01%7C%7Cc81267e5c30d487e7fc408d70b94792d%7C6e0bfede3fcb4518a16565dc14fe5620%7C0&sdata=WYajvR%2BB%2F6M8fIE79qZi1t3ILQ1izlfOX9yMLWbchps%3D&reserved=0 for ideas how to integrate more query languages.

One additional issue is that OpenSearch responses are not STAC compatible AND not GeoJSON compatible. OpenSearch defines a properties field in FeatureCollections, which is not allowed. This is something that should be fixed first.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fradiantearth%2Fstac-spec%2Fissues%2F16%3Femail_source%3Dnotifications%26email_token%3DAC4S3Z5B4EUSS3EQ3I456EDQACDWXA5CNFSM4EAOLNMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2I3JKI%23issuecomment-512865449&data=01%7C01%7C%7Cc81267e5c30d487e7fc408d70b94792d%7C6e0bfede3fcb4518a16565dc14fe5620%7C0&sdata=CwmB0LSIOpTuaptg%2FWxmDS7XCsUANflzqfQ%2FQk8PzT4%3D&reserved=0, or mute the threadhttps://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAC4S3Z3LLRO6FUGGXPFBGRLQACDWXANCNFSM4EAOLNMA&data=01%7C01%7C%7Cc81267e5c30d487e7fc408d70b94792d%7C6e0bfede3fcb4518a16565dc14fe5620%7C0&sdata=9wEoYo%2BZmhXfoCYZvu4mRSIkp52TqU1eIMh3t4unPr4%3D&reserved=0.