Convert old annotation_qc.xml

[cmungall]

A while ago we agreed on xml for specifying our rules http://www.geneontology.org/quality_control/annotation_checks/annotation_qc.xml

These are translated to HTML placed on drupal, and a hyperlink provided for each rule: http://geneontology.org/page/annotation-quality-control#GO_AR:0000014

These URLs are provided in the owltools GAF checks

However, these have suffered some bitrot:

• the HTML generation does not work well with drupal
• the EBI ruleset has diverged cc @tonysawfordebi See also #123
• some rule implementations are SQL
• we never really added to these
• we want to use the same system for noctua/lego rules

We should switch to the same system as https://github.com/geneontology/go-site/tree/master/metadata/gorefs

We can have a yaml structured blob inside human-readable markdown.

TODO

• [ ] decide on rough format
• [ ] convert xml to md or json (preserve IDs?)
• [ ] ensure EBI set covered
• [ ] rewrite owltools GAF reports
• [ ] make some rules executable (will discuss this with @balhoff - consider using SPARQL or SHACL)

[cmungall]

Currently on a branch, can be viewed here for now:

https://github.com/geneontology/go-site/tree/qc-rules/metadata/rules

[kltm]

I think it's (more than) a little weird to have a YAML blob in the markdown--this is just the creation of another format. It would be also just much that harder to parse and control.

I think just straight YAML, with a large text/markdown blob as the final field would be more appropriate. YAML very much supports extended and easy-to-manage text fields.

[kltm]

E.g. > and | field controls.

[cmungall]

See also #287

[cmungall]

We need ASAP a quick view over all rules. I can do a first pass at the script.

[cmungall]

Some of the reasons for doing it this way:

• tried and tested with the OBO Foundry system (works well with jekyll)
• this is a documentation format first, with metadata embedded
• default rendering works well for humans
• extraction is trivial for computational purposes

On 6 Dec 2016, at 7:02, kltm wrote:

I think it's (more than) a little weird to have a YAML blob in the markdown--this is just the creation of another format. It would be also just much that harder to parse and control.

I think just straight YAML, with a large text/markdown blob as the final field would be more appropriate. YAML very much supports extended and easy-to-manage text fields.

-- You are receiving this because you were assigned. Reply to this email directly or view it on GitHub: https://github.com/geneontology/go-site/issues/282#issuecomment-265171958

[kltm]

You wrote it and it's here, so that counts for a lot. However:

• is there a validator for this new format?
• the jekyll connection is really about tying it to github at some level--I don't think you'd want to start bringing on ruby dependencies otherwise
  • looking at https://raw.githubusercontent.com/geneontology/go-site/qc-rules/metadata/rules/gorule-0000002.md, it is a yaml blob and html, not even using markdown conventions, with hard-coded classes that I'm guessing are somewhere in the jekyll config? that's not going to be great to work with long term

It seems to be that the proposed format is at the same time not very good for neither computers (custom format, depends on an infrastructure) nor humans (embedded format, unknown classes, etc.).

[cmungall]

The html is a direct translation of the old qc xml, we should replace with md for ease of editing (but either works for now)

On 13 Dec 2016, at 20:16, kltm wrote:

You write it and it's here, so that counts for a lot. However:

• is there a validator for this new format?
• the jekyll connection is really about tying it to github at some level--I don't think you'd want to start bringing on ruby dependencies otherwise
  • looking at https://raw.githubusercontent.com/geneontology/go-site/qc-rules/metadata/rules/gorule-0000002.md, it is a yaml blob and html, not even using markdown conventions, with hard-coded classes that I'm guessing are somewhere in the jekyll config? that's not going to be great to work with

It seems to be that the proposed format is at the same time not very good for neither computers (custom format, depends on an infrastructure) nor humans (embedded format, unknown classes, etc.).

-- You are receiving this because you were assigned. Reply to this email directly or view it on GitHub: https://github.com/geneontology/go-site/issues/282#issuecomment-266938510

[cmungall]

• validator: 5 mins work for syntactic validator (already done for obo site). For schema validation, can keep using kwalify (but would rather switch to something more standard)
• no ruby dependencies unless we run jekyll
• as noted above the HTML came from the original XML. Github markdown is fine with it but we should nevertheless pandoc it into md
• a bit easier to edit the md when it's not embedded in YAML but YMMV

Will be easy to flip it all into md-in-yaml if we ever need to