support AI-readiness data checks in data quality engine

[mbjones]

For the FAIR Data Quality check engine (issue #328), consult the ESIP AI-Readiness Checklist for a list of data quality checks that the community has felt are important for assessment for prep for ML tooling readiness. See:

• AI-Readiness Checklist
  • ESIP Data Readiness Cluster (2022). Checklist to Examine AI-readiness for Open Environmental Datasets. ESIP. Online resource. https://doi.org/10.6084/m9.figshare.19983722.v1
• ESIP announcement: https://www.esipfed.org/merge/collaboration-updates/checklist-ai-ready-data
• ESIP Data Readiness cluster: https://wiki.esipfed.org/Data_Readiness

I propose that these would be a good candidate set of checks that have already been vetted by ESIP and would be useful way to vet the data quality engine. Maybe it would be it's own suite?

[mbjones]

Also see the Analysis Ready Data (ARD) standards from CEOS:

• CEOS-ARD: https://ceos.org/ard/
• For example, here's the CEOS ARD standard for surface reflectance data:
  • https://ceos.org/ard/files/PFS/SR/v5.0/CARD4L_Product_Family_Specification_Surface_Reflectance-v5.0.pdf

There is a new OGC working group focused on ARD data standards: https://www.ogc.org/press-release/ogc-forms-new-analysis-ready-data-standards-working-group/

[jeanetteclark]

I reformatted the AI Readiness checklist into a csv with a column for whether the "check" could actually be implemented in an automated way. My values in that column is a best guess, first instincts kind of answer. Based on the list I identified the following checks that are already implemented:

• Is there contact information for subject-matter experts?
• Is there a clear data license?
• Is the license standardized and machine-readable (e.g. Creative Commons)?
• Is it available in at least one open, non-proprietary format?
• Is there a comprehensive data dictionary/codebook to describe parameters?
• Does it include details on the spatial and temporal extent?

The following checks could be easily implemented:

• Have null values/gaps been filled?
• What is the timeliness of the data?
• Is there quantitative information about data resolution in space and time?
• Is the provenance tracked and documented?
• Is the data dictionary standardized?
• Do the parameters follow a defined standard?
• Are parameters crosswalked in an ontology or common vocabulary (e.g. NIEM)?
• What is the file format?
• Is it machine-readable?
• Has the data been anonymized / de-identified?

The rest either are not applicable, or would be difficult/impossible to implement.

ML-checks.csv