FRQ: "Check my Content Dictionary" utility on openmath.org

lars-hellstrom commented 7 years ago

Just throwing down an idea in writing…

It would be cool if we had some sort of "Check my Content Dictionary" utility on openmath.org. I'm thinking a UI consisting of a web form, where you can specify an .ocd file to upload, and then press a big shiny button "Check my Content Dictionary". (Uploading doesn't mean we store it, just that we process it.)

Version 0.1 of this utility could simply try validating what was specified against the RNG schema, and respond validates/doesn't validate. But the idea is to add functionality gradually.

One thing that probably would be appreciated is to generate an XHTML rendering of the CD, as we have for those stored on the website. -- Plus, that might give an impulse to get anywhere with some system for notation specifications.
We could also check other things, e.g. does the file seem to contain a license statement and similar more social things, although that is probably more work to implement, so not a priority.
I think it would be pedagogical for the utility to (in its responses) explain what it did. E.g. the way I've found is that I run xmllint --relaxng omcd2.rng myfile.ocd (though there could be better ways); for this the XML-illiterate would probably want to know where to find xmllint (I dunno, it came with the OS) and where to find omcd2.rng (provide link). That kind of thing.

First order of business will probably be to figure out where such a thing can be hosted.

kohlhase commented 7 years ago

I can host anything you (and others) can build, as long as it is almost automatic (i.e. does not need a lot of maintainance in the long run).

davidcarlisle commented 7 years ago

The original cocoon based openmath.org website used an xslt based CD validator on all uploaded contrib cds and only added them if they passed, mailing the results back if they failed. omvalidate.xsl is still in the CDs repo, however gh-pages just offer a static site and client-side xslt is not really trendy these days so I re-wrote (most of) the validator in javascript and added an editable text and/or file drop interface.

It's not linked from anywhere but you can try it out at

https://openmath.github.io/cd/check.html

incidentally if anyone understands the gh-pages/jekyll setup how come I need the .html there but

https://openmath.github.io/cd/arith1

in the same directory serves the same content as

https://openmath.github.io/cd/arith1.html

?

davidcarlisle commented 7 years ago

the test validator is now available at

https://openmath.github.io/cd/check

apparently it just takes a while for gh-pages to set up the redirects.

It now accepts CD , STS or CD Group files.

kohlhase commented 7 years ago

Great work. Now we should integrate it into the web site, and link to it. I think the best way do do that is to make it into a mardown file check.md, adding something like

---
layout: page
title: Validity Check OpenMath Content Dictionaries
---

at the top. That should generate all the boilerplate stuff. I am not sure whether the extensive <head>...</head> makes a problem.

davidcarlisle commented 7 years ago

I think the standard site page is too narrow, this page is just two inline-block divs so it would adjust but it works best side by side rather than one above the other I did wonder about having a two-column version of the layout with the blue navigation sidebar instead shown as a wider top navigation section so there is space to get the two columns underneath.

Also of course it's much easier to test/debug this if it's html as you can test it straight off the local filesystem without having to check it in or run jekyll locally.

David

On 30 July 2017 at 11:53, Michael Kohlhase notifications@github.com wrote:

Great work. Now we should integrate it into the web site, and link to it. I think the best way do do that is to make it into a mardown file check.md , adding something like

layout: page title: Validity Check OpenMath Content Dictionaries

at the top. That should generate all the boilerplate stuff. I am not sure whether the extensive ... makes a problem.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/OpenMath/OpenMath.github.io/issues/27#issuecomment-318893796, or mute the thread https://github.com/notifications/unsubscribe-auth/ABNcAr32NLHoskmL7lXs3eXiyp4hF-P7ks5sTGCRgaJpZM4Obw46 .

lars-hellstrom commented 7 years ago

This is indeed looking good (although longer term we shouldn't forget about the didactical aspect of it — explaining what the tool did!).

But I think I've encountered a bug. I tried it with a CD I'm working on, a fragment of which is

<CDDefinition>
  <Name>M&#x00F6;bius</Name>
  <Role>application</Role>
  <Description>This symbol takes four complex numbers a, b, c, and d such that ad&#x2260;bc as arguments, and constructs the M&#x00F6;bius transformation which maps z to (az+b)/(cz+d). This M&#x00F6;bius transformation is an invertible map from the Riemann sphere to itself.</Description>

and I've validated it against the RelaxNG schema, so it should be correct. However, the web validator says:

CDDefinition:

Name: Möbius (Invalid text)

Role: application

Description: This symbol takes four complex numbers a, b, c, and d such that ad≠bc as arguments, and constructs the Möbius transformation which maps z to (az+b)/(cz+d). This Möbius transformation is an invertible map from the Riemann sphere to itself.

I can't reproduce it here (that I know), but the Name line is coloured bright red to really highlight that there is an error. However, checking the OM standard, a symbol name is defined as an NCName, and according to the XML specification ö (ö) is a NameChar (a NameStartChar, even). Therefore I think this should work.

davidcarlisle commented 7 years ago

On 30 July 2017 at 22:35, lars-hellstrom notifications@github.com wrote:

This is indeed looking good (although longer term we shouldn't forget about the didactical aspect of it — explaining what the tool did!).

yes partly of course that's one benefit of a custom validator like this in that the error message and checks can be tailored to be more informative,

But I think I've encountered a bug.

since I just wrote it in the last couple of days and it's not finished, no behaviour is a bug:-) Actually that's a documented deficiency

// ascii version for now var omNameRE= new RegExp(/^\s[a-zA-Z][a-zA-Z0-9.-_]\s*$/);

I left that for now as I was concentrating on getting the structural checking interleaved with the html presentation correctly.

I need to read up a bit to see if there is any cross-browser javascript that makes checking for the XML element Name production easier. Doing it fully compliantly just with regex would be a bit of a pain as javascript strings are not unicode code points but utf16 pairs, I agree something needs to be done before it's advertised though.

If a full check turns out to be excessively complicated, a better check would be to invert the definition, so fail the name if it contains ascii punctuation like $ or ^ but let everything else through, that is if it says it finds an error it should be an error even if it can not find all errors.

I agree it should not do what it's doing now, which is report valid cases as error.

davidcarlisle commented 7 years ago

@lars-hellstrom Möbius should work now (and Möbius)

davidcarlisle commented 7 years ago

It's a lot faster now (I avoided so many page reflows) and fixed a few more issues, we could probably link to it on the main site unless @lars-hellstrom has managed to break it with real CD examples?

davidcarlisle commented 7 years ago

I added a markdown wrapper page so looks more integrated into the site (wording can not doubt be improved)

https://openmath.github.io/validation/

kohlhase commented 7 years ago

nice.

kohlhase commented 7 years ago

I think this should close this issue.

OpenMath / OpenMath.github.io

FRQ: "Check my Content Dictionary" utility on openmath.org #27

layout: page title: Validity Check OpenMath Content Dictionaries