Tools to validate standards compliance

[dlebauer]

We need to run tests that ensure the data products meet data and metadata format specifications.

Description

ARPA-E milestone for Nov 2017 is to have tests that data products match published metadata standards

Details

• would prevent issues with changing source meta-data

• could be used to ensure correct data format from providers

• a test for extractors

• BETYdb has a solid suite of tests; these may not need to be extended other than capturing errors consistently with other tests

Completion Criteria

• [ ] discuss
• [ ] break into manageable chunks and create new issues

[dlebauer]

Examples for external standards compliance

• cfchecker --> CF compliance
• OGC --> http://www.opengeospatial.org/compliance

Also need to add internal compliance checks

Identify first class data products --> what need to be complied

For example, hyperspectral workflow has three products:

1. level 1: cubes clipped to plot of reflectance values at full resolution, CF + OGC compliant
2. level 2: cube clipped to plot of indices and masks at full resolution, CF + OGC compliant
3. level 3: plot level means of each index inserted into BETYdb

For FLIR, RGB, other images

1. level 1: geoTIFF clipped to plot, OGC compliant
2. level 2: plant/soil mask
3. level 3: traits in BETYdb

need to go through this for ea. data product

[dlebauer]

next steps:

• identify first class data products
• for these, compile current metadata outputs from extractors
• map to OGC, CF, etc
• determine what compliance means