review, clarify and update the MathML tests

[mgylling]

Review, clarify and update (where needed) the MathML tests to make sure all pass conditions are 100% clear to non-MathML experts, especially with regards to pass/fail on existing MathML implementations.

[OriIdan]

Is there anything you see that should be changed?

[pkra]

Hi @OriIdan. After F2F discussion with @mgylling I'm working on a general update. The main goal is to better reflect the purpose of the EPUB test suite (it's not called "the MathML test suite" after all ;-) ).

I'm drafting a first update and will comment on the rationale when it's done.

[pkra]

I've created a first draft for an update at http://codepen.io/pkra/full/OPjzqa.

Right now, I've mostly updated the MathML code. Here are some thoughts on the revamped tests.

• The problem:
  • No browser or polyfill supports (even close to) all of Presentation MathML 3 (except perhaps MathPlayer on IE<10)
  • The original batch of tests tested a very limited coverage of MathML
  • they passed too easily and therefore did not really provide meaningful information
  • The second batch of tests (submitted by myself) were designed to show more realistic use cases
  • a side effect was that no renderer could pass all tests
  • i.e., everyone was set up to fail -- which is similarly not ideal.
• Rationale for the revision
  • the test suite should provide a small selection of tests (not all 1675 from the MathML test suite)
  • this means they won't test everything and that the tests need to be a decent (and always debatable) representation of MathML features.
  • in other words, the tests should cover areas of MathML support that are considered releveant for EPUB3
  • the tests should essentialy test if reading systems have done their homework, i.e., using state of the art of their platforms.
  • if a reading system does everything possible to support MathML, then it should pass the tests.
  • if we get back into the situation of "everyone passes", we can push the envelope with more advanced tests.
• Rationale behind the changes
  • new MathML styling tests
  • it is actually required :-) (as opposed to CSS)
  • it is the more common use case for styling MathML, especially out of publishing workflows
  • it offers an useful comparison with the CSS test
  • I cheat on the border (i.e., use CSS) because the natural MathML solution (menclose) does not officially include dotted borders; this could be worked around using mtables.
  • updated CSS tests:
  • one combined test since this is optional
  • changed tests to move away from element-based styling.
    • it's best practice in a world that requires polyfills on the two leading browser engines
    • polyfills often cannot re-use MathML elements (they have to work harder); element-agnostic styling will (largely) work.
  • add stretchy tests (roots, parentheses)
  • these are important basic features for MathML renderers
  • they are easily checked.
  • Changed mtable test (from col/rowspan to alignments)
  • simplified test to be more transparent to testers
  • test attributes that are more important in real world content
  • a good baseline test: any HTML5 renderer should be abel to have mtables at least be as powerful as regular (HTML/CSS) tables
  • it's low hanging fruit, renderers can catch up easily leading to improved scores
  • Changed BiDi / Arabic test
  • easier for testers
  • no longer "unpassable"
  • updated Elementary Math test
  • additional example
  • higher hanging fruit; reading systems should be able to pass this but it requires some additional work (e.g., on Gecko, you can use this xslt).
• Help for testers
  • reference images for everything
  • better explanations for what to look out for (math rendering can still be subtle and reference images will not in themselves be able to explain what layout differences are acceptable, i.e., differences in typographical style)
  • optionally: add additional documentation on what the expected results on various MathML rendering engines will be.

Does this sound reasonable?

[pkra]

Just a quick bump. If noone objects, I'll work towards a PR.

[mgylling]

Hi Peter, sorry for late reply. I will review within the next few days. /markus

Peter Krautzberger mailto:notifications@github.com 2 Feb 2015 21:56

Just a quick bump. If noone objects, I'll work towards a PR.

— Reply to this email directly or view it on GitHub https://github.com/IDPF/epub-testsuite/issues/83#issuecomment-72535892.

[pkra]

No problem. I'll wait with a PR then.

[pkra]

Just FYI, I've updated the math-styling test to be "pure" MathML.

[mgylling]

Hi Peter, LGTM. Please go ahead with the PR. FYI we will be briefing the testing group on upcoming changes (and also request beta testing of those changes) on a call on the 9th Feb at 18 UTC, wb great if you could join...

Peter Krautzberger mailto:notifications@github.com 4 Feb 2015 12:18

Just FYI, I've updated the math-styling test to be "pure" MathML.

— Reply to this email directly or view it on GitHub https://github.com/IDPF/epub-testsuite/issues/83#issuecomment-72838439.