Discuss how to put in CEDA pipeline and record the QC output and preserve it

[agstephens]

• how to put into ingest pipeline?
• we need to store a report
• we need to respond to the results appropriately

Tagging @gap736uk

[agstephens]

Thoughts and ideas:

• would be useful to give output that says how many PASSES there were as well as ERRORS:
  • build confidence
• can we improve these kinds of errors (e.g. "invalid" is not helpful):

  
  Global attribute: source

• Required 'source' global attribute value is invalid.

Would be better to say:

* Required 'source' is ...invalid. You provided: "foo baa".

• workflow:

  • can we automate?
  • before ingestion, run the checks:
  • if no errors: ingest
  • if errors: mail the Data Support Team

• could we have a quality tick on a Dataset record?

  • i.e. All data in this archive has passed QC checker XYZ (link)

• user interfaces (adjust amf-checker script):

  • [ ] Show WPS interface
  • [ ] Show full verbose output wrapper
  • [ ] Show reduced output wrapper
  • [ ] Show JSON output

• Recording checks:

  • could dump JSON output to Elasticsearch