nasa / cmr-stac

Other
43 stars 21 forks source link

Which item properties get exposed from original metadata? #102

Open scottyhq opened 3 years ago

scottyhq commented 3 years ago

Does this utility only convert required STAC Item metadata, or is it possible to expose additional properties?

For example, a direct search with the ASF Vertex API looks like this: # https://search.asf.alaska.edu/#/?dataset=SENTINEL-1%20INTERFEROGRAM%20(BETA)&polygon=POLYGON((-122.3525%2041.365,-122.0961%2041.365,-122.0961%2041.5252,-122.3525%2041.5252,-122.3525%2041.365))&zoom=9.044887353337925&center=-123.314132,41.235575&resultsLoaded=true&granule=S1-GUNW-A-R-035-tops-20200304_20200203-020818-42666N_40796N-PP-0a7f-v2_0_2&mission=S1%20I-grams%20(BETA)%20-%20Northern%20CA&beamModes=slc&polarizations=VV&flightDirs=Ascending&path=35-35&maxResults=1000&productTypes=GUNW_STD

I'm not sure how STAC Extensions like the SAR extension in this case would be incorporated here, but note some additional very useful search terms such as polarizations=VV&flightDirs=Ascending. These properties are not exposed in the STAC item metadata and therefore not searchable:

https://cmr.earthdata.nasa.gov/stac/ASF/collections/C1595422627-ASF/items/G1714418970-ASF

A "metadata" asset is linked though, which is an XML file with a bunch of additional information.

matthewhanson commented 3 years ago

@scottyhq It's possible to expose additional properties, if we can map CMR fields to STAC properties.

The question I have is are the fields standardized or are the up to the provider and could be arbitrary? In the case of the SAR, is polarizations andflightDirs` common for all SAR data in CMR?

We can still expose them either way, but if they are unique to a provider of course they would only work for that one collection. This would be fine, but over time if we keep adding fields to map that list could get fairly large, and still not work for many collections.

In addition to the two above, are there other fields you'd like to see added?

scottyhq commented 3 years ago

thanks for the reply @matthewhanson !

The question I have is are the fields standardized or are the up to the provider and could be arbitrary? In the case of the SAR, is polarizations and flightDirs` common for all SAR data in CMR?

I don't know the answer to this, hopefully someone more knowledgeable about CMR can answer. At least for ASF DAAC (which is the go-to place for SAR data) the common metadata search "Active Keywords" are documented here https://asf.alaska.edu/api/.

In any case here is an abbreviated table that maps commonly used DAAC API keywords to STAC v1beta keywords. It appears the URL in my original comment is just for the VERTEX GUI interface, so these differ slightly:

DAAC STAC
absoluteOrbit sat:absolute_orbit
beamMode sar:instrument_mode
flightDirection sat:orbit_state
lookDirection sar:observation_direction
platform platform
polarization sar:polarizations
processingLevel sar:product_type
relativeOrbit sat:relative_orbit

Hope that helps!

scottyhq commented 3 years ago

@matthewhanson @jaybarra - just wanted to re-raise this issue. Is there a standard process to adding the additional fields in the table above? It's really important to be able to search and filter on additional fields for SAR in addition to just bbox and time. Otherwise, there are just too many items to work with!

https://cmr.earthdata.nasa.gov/stac/ASF/collections/SENTINEL-1A_DP_GRD_HIGH.v1/items