Scenario 1B Progress

[kwilcox]

Progress: 1B Core Variables

Successes

• Able to query MMI for a list of IOOS Core Variables
• Able to obtain CF Standard names through MMI for all IOOS Core Variables
• Able to search CSW servers on CF Standard names to get back a summary of offerings

  Easy Challenges: Identified and Fixed

• The CSW server at data.noaa.gov was not working: https://github.com/ioos/system-test/issues/130. Problem was fixed and the CSW server is now operational.

  Longer-term Challenges Identified

• Programmatic access to the IOOS Core Variable is not easy: https://github.com/ioos/system-test/issues/128
• CF alias terms are not available in MMI using a SPARQL query: https://github.com/ioos/system-test/issues/129
• OWSLib does not automatically paginate returned CSW records. Would be nice if it did this automagically: https://github.com/ioos/system-test/issues/126

[emiliom]

Is there a notebook that goes with this? I'm intrigued by this success:

Able to obtain CF Standard names through MMI for all IOOS Core Variables How did you end up getting the linkage from IOOS Core Variables to CF standard names?

Such a mapping still doesn't exist on MMI, AFAIK (#129 and https://github.com/ioos/vocabularies/issues/1). The relevant mappings that exist are IOOS Core Variables to IOOS Parameters, and IOOS Parameters to CF standard names. I assume SPARQL would allow a double hop (IOOS Core Variables > IOOS Parameters > CF standard names); it'd be fun to see the code that does it, though.

[dpsnowden]

See PR #131

[emiliom]

Thanks, @dpsnowden ! Very cool. But, strictly speaking, that notebook is not actually using the IOOS Core Variables to query for CF standard names. The first block uses rdflib to get a pandas DataFrame of the IOOS Core Variables. Cool stuff, @kwilcox . But the second block, "Query MMI for CF standard names related to the IOOS Core Variables", is not actually doing what's advertised. Instead, it's finding CF standard names related to the IOOS Parameter Vocabulary, not the IOOS Core Variables. It looks like what's done in the first block with Core Variables is not really reused in the rest of the notebook. Just pointing this out to clarify that the notebook description is misleading: "This notebook looks at the IOOS core variables and uses the Marine Metadata Interoperability SPARQL endpoint to convert them to CF Standard names".

This issue has inspired me to look more closely into the "double hop" (IOOS Core Variables > IOOS Parameters > CF standard names) vocabulary matching I mentioned. I've borrowed from @neaptide's http://www.unc.edu/~haines/orrioos.html and a bit from @kwilcox 's notebook, to expand a notebook I created earlier. I already had a SPARQL query selecting IOOS Parameters matching one IOOS Core Variable term (dissolved_nutrients); I've added a SPARQL query that then selects CF standard names matching the IOOS parameters previously selected (ie, ones that matched the IOOS Core Variable dissolved_nutrients). See the section at the end. Also, @neaptide's https://github.com/nccoos/ioos-vocabulary-viz previously implements some of that mechanism, also in Python, but for a UI. Cool stuff, Sara!