Milestone - Continuous FAIRness evaluation acros the NMDC (0.19)

[ssarrafan]

Monitoring & Evaluation To track the NMDC team’s progress over the three year Renewal, we have developed an initial set of metrics during the Pilot (Appendix 23) and will implement processes for tracking and regular reporting (Milestone 0.16). This monitoring and evaluation (M&E) system is designed to keep NMDC on track with our vision and mission. This M&E system includes goals, stated below, around research impact, infrastructure development, and community engagement. For each of these goals there are a variety of indicators such as usage numbers, datasets available in the portal, bugs reported and resolved, and event attendance. We additionally plan to evaluate and track FAIRness across the entire NMDC program (Milestone 0.17). These metrics will be tracked on a quarterly basis alongside the Agile sprint structure. By tracking these metrics, the NMDC team will be able to monitor our progress and articulate to partners and project managers the evolution and development of NMDC’s progress and aims.

Page 21

[shreddd]

@dwinston - any updates on this to report on?

[dwinston]

draft FAIR Implementation Profile (FIP): https://github.com/microbiomedata/NMDC_documentation/issues/37

[dwinston]

specific to the NMDC Schema (i.e. the semantic resource/vocabulary), would like to note https://github.com/agroportal/fairness (example application: http://earthportal.eu/ontologies/BRGM_UOM (see "FAIR Scores" section)) as an avenue for tracking schema fairness (separate from (meta)data and associated services).

[shreddd]

See also: https://fairplus.github.io/Data-Maturity/

[ssarrafan]

Update draft for the quarterly DOE report 3/29/24: A FAIR Implementation Profile was drafted and a semantic resource/vocabulary resource created in GitHub as an avenue for tracking schema fairness. The team has planned the continuation of FAIR evaluation going forward in the following three phases: Phase 1: Implement CI pipeline that uses fair-test library to evaluate the NMDC repos against FAIR tests and run weekly with alert notification. Phase 2: Customize tests in CI pipeline using Data Maturity principles to evaluate data FAIRness and OFAIRe to evaluate schema. Phase 3: Create dashboards to present the FAIR evaluation.

IF you have any updates, or changes please let me know @shreddd @dwinston @PeopleMakeCulture

[dwinston]

I’d change the first sentence to: A FAIR Implementation Profile was drafted and a FAIRness assessment tool for ontologies, vocabularies and semantic resources was identified as an avenue for tracking schema fairness.

Otherwise, looks good.

[ssarrafan]

I'm moving this to Q2 2025 for the next round of updates.