mweiden commented 5 years ago

Need

User stories:

As a member of Data Operations, I want the ability to perform a check on the quality of metadata as processed by the system, so that I can verify that the metadata is correct.

Example application:

Ensure that all text and ontology ID pairs are correct given the ontology.

Definition of done

There is a way for DataOps to define, submit, and run metadata validation jobs across data that has been submitted to the system such as those described in the example applications.
Information on validation status is surfaced to data ops (and wranglers) a human-readable, easy-to-use way (maybe tracker).
Work with DataOps and wranglers to write and deploy 2 validation checks.

Questions:

Do we want to verify before or after the data is in the system?
- Afterwards
What does verify mean?
- Correct fields
- Matches other data (anything in secondary bundle matches primary bundle)
- Values are correct relative to each other
- Experiment graph doesn’t violate particular conditions

Notes:

jahilton 10:28 AM - I think the higher priority is automated & thorough metadata checking; I think within projects is the scope of metadata validation
Could be integrated into the data tracker; validation jobs can be put into the tracker in a modular form
Could use query service as a source of information for the predicates

Related documents:

Encode’s metadata auditing library - https://github.com/ENCODE-DCC/encoded/tree/dev/src/encoded/audit
DCP wranglers link validation issue - https://github.com/HumanCellAtlas/ranglr/issues/2

jahilton commented 5 years ago

I think the higher priority is automated & thorough metadata checking; I think within projects is the scope of metadata validation

I think what I meant by this was to prioritize pre-submission validation. If something can be checked at ingest (spreadsheet upload), then it should be, especially now as whatever gets ingested is immediately public.