Qt QWebpage Windows and mml3.js question

[kevinhendricks]

Your single-file version works wonderfully with Qt 5.4.2 QWebpage (Qts version of webkit) on Linux and Mac OSX but your 2.5.1 version and official Mathjax 2.6.0 both fail to make the mml3.js extension work on Windows Qt 5.4.2. My guess it is the xslt transformation in mml3.js that is failing. I noticed that Mathjax 2.6.0 has added some Browser check and changed the xsl code to work on Edge and MSIE for Windows. Qt reports Safari Apple Webkit as user agent so it is not using the special msn xslt namespace changes.

Do you have any plans to update this to 2.6.0 anytime soon. Or should I download and install npm and try your grunt approach to creating a single file version of MathJax 2.6.0 so that I can add to the Browser test to take advantage of 2.6.0 new xsl?

Thanks,

[pkra]

Your single-file version works wonderfully with Qt 5.4.2 QWebpage (Qts version of webkit) on Linux and Mac OSX

Glad to hear it's useful.

but your 2.5.1 version and official Mathjax 2.6.0 both fail to make the mml3.js extension work on Windows Qt 5.4.2.

Sorry to hear that.

My guess it is the xslt transformation in mml3.js that is failing. I noticed that Mathjax 2.6.0 has added some Browser check and changed the xsl code to work on Edge and MSIE for Windows.

That would make sense but you mentioned above that 2.6.0 caused issues as well, so I'm not sure. (I'm assuming you mean issues beyond those stated at https://github.com/mathjax/MathJax/pull/1257#issuecomment-138948161).

Also, http://www.qt.io/qt5-4/ seems to indicate that QT WebEngine is Chromium-based (except on mobile devices).

Qt reports Safari Apple Webkit as user agent so it is not using the special msn xslt namespace changes.

Ah, ok.

I noticed that Mathjax 2.6.0 has added some Browser check and changed the xsl code to work on Edge and MSIE for Windows.

That's correct.

Do you have any plans to update this to 2.6.0 anytime soon.

I was planning to make an update next week but feel free to make a pull request if you need it sooner.

There's also #2 I wanted to fix but that can happen later.

[pkra]

Re-reading my posting just now, I'm still not sure how a 2.6.0 build will help with your issue.

[kevinhendricks]

Hi, We use your single version (or are trying to) in Sigil (GPL Epub Ebook Editor) as a local link to try to allow a QWebPage Preview window to show epub authors what their mathml will look like embedded in their epub webpage. I was planning to hack your MathJax.js single to force the msn xslt version of the stylesheet in the mml3.js part of the code, and see if it works to enable mml3 extensions. If so, I plan to send a patch to MathJax to extend the Browser check to include a new Browser (Windows, Apple Webkit, Safari (useragent) detection and add it to the mml3.js code to that special case (Edge, MSIE). If the xslt translation is not the issue, then I do not know what is preventing the mml3 extensions from working but only on Windows.

Hope this explains things a bit better.

[kevinhendricks]

I grabbed MathJax master's unpack directory and modified your Gruntfile.js to do a head of 3241 lines (replacing 3199), and a tail of 29 lines (no change from previous) and ran grunt and it seems to have created a 2.6.0 version of MathJax single for me to play with to see if the mml3 issue is in fact related to xslt. If need be I can convert it back to a separate mml3.xsl file and manually apply it to all mathml sections before loading the page, but I would rather avoid that if possible. Thanks

[kevinhendricks]

Closing this issue. Tried xslt using both namespaces on Windows and no deal. I then looked and Qt WebKit on Windows is not linked to libxml2 or libxslt. So my guess is that XSLT transformations simply do not work on Qt Webkit on Windows with stock Qt builds. That is why the mml3.js does not work. Qt QWebEngine is Chrome webkit based and should work even on Windows but it would require a large rewrite of Sigil to move from Qt Webkit to Qt WebEngine. Not something I am likely to do any time soon. I guess I will have to extract the stylesheet to recreate the original mml3.xsl and then apply it to the source of the ebook page before loading it and invoking MathJax. Sorry for the noise.

[pkra]

Thanks for looking into it yourself!

We use your single version (or are trying to) in Sigil (GPL Epub Ebook Editor) as a local link to try to allow a QWebPage Preview window to show epub authors what their mathml will look like embedded in their epub webpage.

Very cool. I've been following the Sigil development as a lurker but hadn't had a chance to take a closer look at the new epub3 features. But it's always been awesome so thank you for Sigil!

I grabbed MathJax master's unpack directory and modified your Gruntfile.js to do a head of 3241 lines (replacing 3199), and a tail of 29 lines (no change from previous) and ran grunt and it seems to have created a 2.6.0 version of MathJax single for me to play with to see if the mml3 issue is in fact related to xslt.

Thanks for trying it out.

Tried xslt using both namespaces on Windows and no deal.

Ah, sorry to hear that.

I then looked and Qt WebKit on Windows is not linked to libxml2 or libxslt. So my guess is that XSLT transformations simply do not work on Qt Webkit on Windows with stock Qt builds. That is why the mml3.js does not work.

I see. That's too bad (and somewhat strange to only affect Windows).

Qt QWebEngine is Chrome webkit based and should work even on Windows but it would require a large rewrite of Sigil to move from Qt Webkit to Qt WebEngine. Not something I am likely to do any time soon.

Understood. I was just stabbing in the dark.

One more thing worth trying would be to remove the Content MathML extension. There have been conflicts in the past among the two.

I'm re-opening so that I'll remember to file something upstream (and maybe finally get myself to write a JS version of that xslt...).

Final question if you have the teim: what kind of issues did you actually run into?

[kevinhendricks]

The Qt version of Webkit has had mathml disabled in its build. The chrome version of webkit (QWebEngine) has removed all mathml support. Thus Qt does not make a webkit/webpage widget that actually supports true html5 (which includes mathml). To support epub3, Sigil must somehow support mathml. There is an official epub3 Testsuite epub which incorporates a mathml test page and it tests both mlongdiv and mstack, mcarries, so you need them to work to pass the mathml test.

On Linux and Mac OS X everything works perfectly with MathJax and Qt.

On Windows with the damn Qt WebPage (Qt webkit), there are two issues. The first is that the mml3 extensions do not work (which prompted the above bug report), and the second is that where other mathml is used, the svg version of the formula appears incredibly small. The formula is correct but the resulting scale for the SVG, makes the font appear about 1/3rd of the size of the surrounding text. But again this is only on Windows.

Any ideas of where/how to control the scale factor on the svg produced by MathJax? Or even how the scale is determined? Any way to config MathML settings to increase the scale to make the equations appear larger on the screen?

BTW: a pure javascript version that does not use XSLT would be wonderful! Getting libxml2 and libxslt to compile on Windows is a real headache. Chrome also has issues with its XSLT implementation not handling import and a few other nits. Relying on xslt transformations seems to have gone the way of the wind these days.

What I really need is MathJax.js single converted to python 3.5! So if you are looking for a project, I've got just the one for you!

Right now, in CodeView we leave the pure mathml. We have a Preview mode in which we copy the current code out of the CodeView window, check for math tags, if they exist, inject a local copy of MathJax.js- single, and show the page to the user. A Timer then every few seconds updates this page as the code in the CodeView page is edited by the user. Therefore we need the MathJax script to run quite quickly for any of this to work. Using your MathJax single version as a local file seems to make it work the best. Unfortunately, it simply is not working on Windows. We need to support all 3 main platforms equally (Windows, Linux, and Mac OS X), so I have reached a hurdle until I figure out how to get both the size increased and mml extensions to work for Qt Webkit on Windows.

Thanks,

Kevin

[kevinhendricks]

BTW, At https://www.qupzilla.com/ you will find a Qt based free lightweight Browser. Their github master has been moved to Qt QWebEngine but their current release binaries for version 1.8.9 is built with Qt webkit on all platforms. I use this QUpZilla browser for testing Qt webkit functioning on Windows (as rebuilding all of Sigil on Windows is a real pain).

If I inject MathJax into a page with mathml including mlongdiv tests and use this browser on Windows, mlongdiv fails (the mml3 xslt transformation fails for some reason) and the equations you do see are much too small. The Mac and and Linux versions of this lightweight browser (v1.8.9), pass the all tests with flying colors.

I then followed the link in mml3.js back to grab David's mml3mj.xsl file and used the following hardcoded python3 script with lxml to preprocess .xhtml page to a new junk.xhtml test page.

import sys
import os
import lxml.etree as ET

xml_filename = "/Users/kbhend/Desktop/content-mathml-001.xhtml"
xsl_filename = "/Users/kbhend/Desktop/mml3mj.xsl"
dom = ET.parse(xml_filename)
xslt = ET.parse(xsl_filename)
transform = ET.XSLT(xslt)
newdom = transform(dom)
newdoc = ET.tostring(newdom, pretty_print=True)
if type(newdoc) == str:
    newdoc = newdoc.encode('utf-8')
with open('junk.xhtml', 'wb') as f:
    f.write(newdoc)

The resulting junk.xhtml file when loaded in QupZilla browser on Windows, now passes mlongdiv and mstack/mcarries tests on Windows. Again, the resulting svg images have the wrong scale (much too small) but at least it works.

I think this confirms that Qt webkit on Windows is failing is some place in the mml3 xslt transformation.

This is so frustrating. I guess I will manually apply the mml3 transform for Windows before loading the page. I see MathJax has gotten special dispensation from the author of mml3mj.xsl (David?) to use it. I will have to contact him to see if I can use his mml3mj.xsl file in Sigil (Sigil is GPL).

If you want a testcase to play around with the QupZilla lightweight browser and MathJax, let me know and I would be happy to extract the epub3 mathml test page and its associated images and post it for you to make testing easier.

Kevin

[kevinhendricks]

I found the svg config area in the Gruntfile.js and will modify it to set the proper svg scale. Kevin

[pkra]

There is an official epub3 Testsuite epub which incorporates a mathml test page and it tests both mlongdiv and mstack, mcarries, so you need them to work to pass the mathml test.

Yes, I'm to blame, I think.

On Windows with the damn Qt WebPage (Qt webkit), there are two issues. The first is that the mml3 extensions do not work (which prompted the above bug report),

IIUC your later comments you mention the mlongdiv example from the epub3 test. I'm guessing you see MathJax Processing Errors. Do you see an error for the RTL example?

and the second is that where other mathml is used, the svg version of the formula appears incredibly small. The formula is correct but the resulting scale for the SVG, makes the font appear about 1/3rd of the size of the surrounding text. But again this is only on Windows.

Some screenshots would be good (and maybe a bug upstream at MathJax . It sounds like a missing viewport meta tag, cf. http://docs.mathjax.org/en/latest/output.html#viewport-meta-tag.

If not then this sounds like a bug which we'd love to hear more about upstream at MathJax/MathJax.

Any ideas of where/how to control the scale factor on the svg produced by MathJax? Or even how the scale is determined? Any way to config MathML settings to increase the scale to make the equations appear larger on the screen?

You shouldn't have to but from later commments, I'm guessing you found http://docs.mathjax.org/en/latest/options/SVG.html.

What I really need is MathJax.js single converted to python 3.5! So if you are looking for a project, I've got just the one for you!

Hah! We'd love to see more math rendering engines out there but MathJax is quite complex. Maybe write a python wrapper around mathjax-node could help.

Right now, in CodeView we leave the pure mathml. We have a Preview mode in which we copy the current code out of the CodeView window, check for math tags, if they exist, inject a local copy of MathJax.js- single, and show the page to the user. A Timer then every few seconds updates this page as the code in the CodeView page is edited by the user. Therefore we need the MathJax script to run quite quickly for any of this to work. Using your MathJax single version as a local file seems to make it work the best. Unfortunately, it simply is not working on Windows. We need to support all 3 main platforms equally (Windows, Linux, and Mac OS X), so I have reached a hurdle until I figure out how to get both the size increased and mml extensions to work for Qt Webkit on Windows.

Thanks for the background information. It's a common enough use case, I think, even though we realize performance is not good enough for blindly updating an entire document frequently.

At https://www.qupzilla.com/ you will find a Qt based free lightweight Browser. Their github master has been moved to Qt QWebEngine but their current release binaries for version 1.8.9 is built with Qt webkit on all platforms. I use this QUpZilla browser for testing Qt webkit functioning on Windows (as rebuilding all of Sigil on Windows is a real pain).

I'll try to find time to check it out. We've seen erratic issues reported at MathJax about QTWebKit in the past, so I'm not surprised there are more issues.

I then followed the link in mml3.js back to grab David's mml3mj.xsl file and used the following hardcoded python3 script with lxml to preprocess .xhtml page to a new junk.xhtml test page. The resulting junk.xhtml file when loaded in QupZilla browser on Windows, now passes mlongdiv and mstack/mcarries tests on Windows. Again, the resulting svg images have the wrong scale (much too small) but at least it works. I think this confirms that Qt webkit on Windows is failing is some place in the mml3 xslt transformation.

Thanks for going the extra mile. I would agree.

I see MathJax has gotten special dispensation from the author of mml3mj.xsl (David?) to use it. I will have to contact him to see if I can use his mml3mj.xsl file in Sigil (Sigil is GPL).

David Carlisle has licensed most of his code under every imaginable open license -- he's amazing.

If you want a testcase to play around with the QupZilla lightweight browser and MathJax, let me know and I would be happy to extract the epub3 mathml test page and its associated images and post it for you to make testing easier.

Thanks, no need for that.

[kevinhendricks]

Yes, I'm to blame, I think.

Ah! It is nice to know who to send the dogs after! ;-)

For our Sigil 0.9.3 release, we are going to include a MathJax - single (but updated to 2.6.0 and with SVG scale set to 175% in the config section). This seems to have done the trick. Windows versions of Sigil simply will not support mlongdiv and mstack and so will show the "Unknown node type: mlongdiv" error box. At least until to some future time when I can figure out how to deal with this.

I tried using safari_wrench, and googles ajaxslt as they claim to have pure javascript implementations of xslt but both are too incomplete in their support of xslt to deal with the mml3mj.xsl. Qt's own QXmlQuery does not support enough of xslt to handle that particular stylesheet either (according to their XSLT 2.0 doc page). I really do not want to have to invoke Sigil's embedded Python 3.5 interpreter just to fire up lxml/lxml2/libxslt to deal with the transformation needed to be done every few seconds when a page with mathml is active. So Windows users will have to "suffer" for a while until I can figure something else out. It is so sad that Google decided to pull its mathml support, and that Qt decided to disable Apple's webkit based mathml implementation. If someone would only create a Gecko widget or a FireFox webkit clone widget for Qt that actually worked. Thanks for all your help.

Kevin

[pkra]

Windows versions of Sigil simply will not support mlongdiv and mstack and so will show the "Unknown node type: mlongdiv" error box.

Thanks! That indicates that the mml3 extension does not pre-process anything and just passes it through. I'll try to find the time to look into it.

[pkra]

Closing in favor of upstream issue.