Closed ronaldtse closed 5 years ago
Assuming there is a functioning LaTeX to OOXML converter, otherwise Word won't work.
@opoudjis I heard stem=latexmath
doesn't require LaTeX. If latexmath can be converted into AsciiMath, then it's all good?
And trust the conversion chain LaTex > AsciiMath > MathML > OOXML? You're an optimist, Ronald; I'm not :)
Well, any issues in this chain are bugs :-)
This sample is from the OGC CDB 1.1 standard. Here's the full code:
Compile from the terminal using `asciidoctor -a stem=latexmath --safe -a data-uri examples.adoc`
\[Texture\ Dimension = \ 2^{n}\ \times \ 2^{m}\]
\[Number\ of\ Mipmaps = \max\left( n,m \right) + 1\]
\[Texture\ Size = 2^{W}\ \]
\[C = C_b \cdot (1-A_n)+C_n \cdot A_n\]
\[\delta X = \rho _t \cos (lat) \delta lon \]
Below is an example of matrix embedded in side a table
[cols="3,1",]
|==========
a|
\[ M =
\begin{bmatrix}
-\sin λ_0 & \cos λ_0 & 0 \\
-\sin φ_0 \cos λ_0 & -\sin φ_0 \sin λ_0 & \cos φ_0 \\
\cos φ_0 \cos λ_0 & \cos φ_0 \sin λ_0 & \sin φ_0
\end{bmatrix}
\]
|(eq. A‑9)
|==========
Another example.
[cols="3,1",]
|==========
a|
\[ M_x =
\begin{Bmatrix}
1 & 0 & 0 \\
0 & -1 & 0 \\
0 & 0 & -1
\end{Bmatrix}
\]
|(eq. A‑12)
|==========
And desired output:
So Asciidoctor handes latexmath
with https://github.com/asciidoctor/asciidoctor-mathematical , which is based off https://github.com/gjtorikian/mathematical.
The https://github.com/gjtorikian/mathematical gem supports MathML output, which is perfect for us. So perhaps we can fork asciidoctor-mathematical to support MathML output, or create an extension to bind to mathematical
.
Won't use asciidoctor-mathematical, that uses only the image output of mathematical. I am going to go straight to Mathematical, and process latex stem expressions one at a time, just like AsciiMath is being processed now.
Note that the installation of the mathematical gem has a lot of messy dependencies: bundler is not sufficient. You will have to tailor the metanorma installer to deal with it.
https://github.com/gjtorikian/mathematical/issues/87
Refer https://github.com/gjtorikian/mathematical/blob/master/script/bootstrap : I was missing cmake from my environment
The bit of mathematical in turn that is actually needed is https://github.com/gjtorikian/mtex2MML, a Bison grammar converting TeX to MathML. The problem with compiling that is, flex is broken on newer Macs, if you've ever used MacPorts: https://superuser.com/questions/1276478/flex-gives-no-output-on-macos-high-sierra-xcode-9
This code is looking too finicky to use in general redistribution.
if it doesn't work (ran out of time reinstalling macports), consider alternatives, e.g. https://www.gipp.com/wp-content/papercite-data/pdf/schubotz2018.pdf
SnuggleTeX: is an open-source Java library developed at the University of Edinburgh8. The tool allows to convert simple LaTeX expression to XHTML and presentation MathML.
And in fact that paper points to LaTeXML from NIST, it was slowest but apparently the best at quality and consistency.
LaTeXML has a Homebrew formula (on macOS), is also available via RPMs on Linux. It relies on Strawberry Perl's CPAN on Windows.
https://dlmf.nist.gov/LaTeXML/
(There is a Chocolatey package for Strawberry Perl: https://chocolatey.org/packages/StrawberryPerl)
Looks like we should create an extension package called "metanorma-latexmath" to allow latexmath, based on LaTeXML (for example)?
Now that I have reinstalled my old macports, and installed cmake, mathematical installs fine, and converts Latex to MathML just fine. I just don't know whether we can trust it not to do this to other people with old macports installations.
The MathML generated by mathematical (which is of course really mtex2MML) is quite redundant:
m.render('$a \ne b$')
=> {:data=>"<math xmlns='http://www.w3.org/1998/Math/MathML' display='inline'><semantics><mrow><mi>a</mi><mo>≠</mo><mi>b</mi></mrow><annotation encoding='application/x-tex'>a \\ne b</annotation></semantics></math>"}
The semantics
wrapper and the annotation
elements will have to be removed in postprocessing.
@opoudjis that is expected, since the paper says LaTeXML creates the best quality/consistency (least distance) MathML output.
As an experiment with LaTeXML, it works.
Here's the file I used (the contents was copied from the example above):
math.tex
\documentclass{article}
\usepackage{amsmath}
\begin{document}
$$
M =
\begin{bmatrix}
-\sin λ_0 & \cos λ_0 & 0 \\
-\sin φ_0 \cos λ_0 & -\sin φ_0 \sin λ_0 & \cos φ_0 \\
\cos φ_0 \cos λ_0 & \cos φ_0 \sin λ_0 & \sin φ_0
\end{bmatrix}
$$
\end{document}
Then I ran:
latexml math.tex | latexmlpost --cmml - > math.xml
Then the math.xml
is:
<?xml version="1.0" encoding="UTF-8"?>
<?latexml searchpaths="/Users/mulgogi/src/mn-nist-documents"?>
<?latexml class="article"?>
<?latexml package="amsmath"?>
<?latexml RelaxNGSchema="LaTeXML"?>
<document xmlns="http://dlmf.nist.gov/LaTeXML" xmlns:m="http://www.w3.org/1998/Math/MathML" xml:id="Document">
<resource src="LaTeXML.css" type="text/css"/>
<resource src="ltx-article.css" type="text/css"/>
<para xml:id="p1" fragid="p1">
<equation xml:id="S0.Ex1" fragid="S0.Ex1">
<Math mode="display" tex="M=\begin{bmatrix}-\sin λ_{0}&\cos λ_{0}&0\\ -\sin φ_{0}\cos λ_{0}&-\sin φ_{0}\sin λ_{0}&\cos φ_{0}\\ \cos φ_{0}\cos λ_{0}&\cos φ_{0}\sin λ_{0}&\sin φ_{0}\end{bmatrix}" text="M = matrix[[- sine@(Î * » _ 0), cosine@(Î * » _ 0), 0], [- sine@(Ï * _ 0) * cosine@(Î * » _ 0), - sine@(Ï * _ 0) * sine@(Î * » _ 0), cosine@(Ï * _ 0)], [cosine@(Ï * _ 0) * cosine@(Î * » _ 0), cosine@(Ï * _ 0) * sine@(Î * » _ 0), sine@(Ï * _ 0)]]" xml:id="S0.Ex1.m1" fragid="S0.Ex1.m1"><m:math alttext="M=\begin{bmatrix}-\sin λ_{0}&\cos λ_{0}&0\\ -\sin φ_{0}\cos λ_{0}&-\sin φ_{0}\sin λ_{0}&\cos φ_{0}\\ \cos φ_{0}\cos λ_{0}&\cos φ_{0}\sin λ_{0}&\sin φ_{0}\end{bmatrix}" display="block"><m:apply><m:eq/><m:ci>𝑀</m:ci><m:matrix><m:matrixrow><m:apply><m:minus/><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:apply><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply><m:cn type="integer">0</m:cn></m:matrixrow><m:matrixrow><m:apply><m:minus/><m:apply><m:times/><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:apply></m:apply><m:apply><m:minus/><m:apply><m:times/><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:apply></m:apply><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:matrixrow><m:matrixrow><m:apply><m:times/><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:apply><m:apply><m:times/><m:apply><m:cos/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Î</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci>»</m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:apply><m:apply><m:sin/><m:apply><m:times/><m:ci>italic-Ï</m:ci><m:apply><m:csymbol cd="ambiguous">subscript</m:csymbol><m:ci></m:ci><m:cn type="integer">0</m:cn></m:apply></m:apply></m:apply></m:matrixrow></m:matrix></m:apply></m:math></Math>
</equation>
</para>
</document>
So there you can see the LatexMath has been converted into MathML.
NOTE: The \begin{document}...\end{document}
and \usepackage{amsmath)
is necessary for LaTeXML to work.
Well this way is even simpler!
math-only.tex
M =
\begin{bmatrix}
-\sin λ_0 & \cos λ_0 & 0 \\
-\sin φ_0 \cos λ_0 & -\sin φ_0 \sin λ_0 & \cos φ_0 \\
\cos φ_0 \cos λ_0 & \cos φ_0 \sin λ_0 & \sin φ_0
\end{bmatrix}
Command:
cat math-only.tex | latexmlmath --preload=amsmath --pmml=math-only.xml -- -
Output (math-only.xml
):
<?xml version="1.0" encoding="UTF-8"?>
<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="M=\begin{bmatrix}-\sin λ_{0}&\cos λ_{0}&0\\ -\sin φ_{0}\cos λ_{0}&-\sin φ_{0}\sin λ_{0}&\cos φ_{0}\\ \cos φ_{0}\cos λ_{0}&\cos φ_{0}\sin λ_{0}&\sin φ_{0}\end{bmatrix}" display="block">
<mrow>
<mi>M</mi>
<mo>=</mo>
<mrow>
<mo>[</mo>
<mtable columnspacing="5pt" displaystyle="true" rowspacing="0pt">
<mtr>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
<mo></mo>
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
<mo></mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd columnalign="center">
<mrow>
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
<mo></mo>
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mrow>
<mi>cos</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
<mo></mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Î</mi>
<mo></mo>
<msub>
<mi mathvariant="normal">»</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>sin</mi>
<mo></mo>
<mrow>
<mi>Ï</mi>
<mo></mo>
<msub>
<mi mathvariant="normal"></mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
</mtr>
</mtable>
<mo>]</mo>
</mrow>
</mrow>
</math>
The mathematical gem (and specifically its mtex2MML dependency) is also failing to install in Travis, which is breaking metanorma currently:
/home/travis/.rvm/rubies/ruby-2.5.1/lib/ruby/2.5.0/fileutils.rb:1227:in `stat':
No such file or directory @ rb_file_s_stat -
/home/travis/build/riboseinc/metanorma/vendor/bundle/ruby/2.5.0/gems/mathematical-1.6.12/ext/mathematical/lasem/build/liblasem.so
(Errno::ENOENT)
This is an issue somehow related to https://github.com/asciidoctor/asciidoctor-mathematical/issues/30 (there is a long history of liblasem failing to compile). At any rate, we will be using LateXML instead.
Require "brew install latexml" and its Windows counterpart added to metanorma installers. @ronaldtse, can someone else do that?
Yes, two things:
The implementation should "detect" whether latexml is installed in order to support latexmath. If it is not installed and there is a block of latexmath, it should throw an error and abort?
@CAMOBAP795 can you help add "latexml" package to https://github.com/riboseinc/homebrew-metanorma, and also to https://github.com/riboseinc/chocolatey-metanorma/ ? It seems that there is no Chocolatey package for LaTeXML though... It's available via Perl.
@ronaldtse sure could you please create separate repo chocolatey-latexml
Here is https://github.com/riboseinc/homebrew-metanorma/pull/9
@opoudjis can you tell me some short cli command which I can use locally to make sure that latexml
installed and available for metanorma
?
Here you go https://github.com/riboseinc/chocolatey-latexml
@ronaldtse You haven't noticed, but the latexml output is utterly garbling Unicode, and it's not obvious to me how to prevent that. If I can't resolve this, this is useless. Trying to resolve.
@CAMOBAP795 latexml --VERSION
@opoudjis thanks, but I mean if not call it as standalone tool, will metanorma --type ogc test.adoc
call for latexml?
I am currently just doing a system call to latexml from within metanorma-standoc.
I am nowhere near uploading it, because right now its output is so garbled it will not process in XML. So you can't test yet.
@opoudjis ok, thanks for explanation
So just to be on the same page latexml
will be part of metanorma-standoc
(not metanorma-ogc
), right?
Yes, it will be inherited; this will be generic Metanorma functionality.
IF IT WORKS. It renders, but its output is not Unicode, it's some 15 year old ISO-Latin-1 crap right now.
Sorry, but please hold off until I confirm this is usable at all.
@opoudjis np, let me know if you will need some help (because I observed that some latexml
from github don't set output encoding explicitly, but this one https://dlmf.nist.gov/LaTeXML/releases/LaTeXML-0.8.2.tar.gz does)
OK, it outputs UTF-8 fine; it just does not input UTF-8 at all. If I use \lambda
, it renders fine. If I use λ, per Ronald's example, it is a garbled ISO character, which blows UTF-8 up. I will see if I can somehow work around that.
@opoudjis latexml --inputencoding=UTF-8
doesn't help there?
@opoudjis the problem is garbled text in, garbled text out. Strictly speaking, the formula is supposed to use “\lambda”.
It’s just that sometimes people take the shortcut. Even if it fails with garbled input, it’s acceptable.
I'm implementing a UTf8 to Latex converter, so any instances of λ are converted to \lambda. No ruby gems to my surprise, but this will be simple enough.
I will do this within metanorma-standoc for now, but will make separate gem later.
Resolved UTF8 issue; but output on sample is still incorrect.
Working fine now; will publish separate unicode2latex gem.
@ronaldtse please enable appveyor for https://github.com/riboseinc/chocolatey-latexml
Note that this will not work with downstream gems like OGC until the next release of the metanorma-standoc gem.
@ronaldtse Please enable Travis on https://travis-ci.com/metanorma/unicode2latex
@opoudjis how can this can be tested via metanorma-cli
(when
metanorma-standoc
will be release)
I'm asking because I need to write some tests of homebrew package
@opoudjis Travis is enabled for https://travis-ci.com/metanorma/unicode2latex
@CAMOBAP795 Convert the following file (from the metanorma-standoc rspec:
= File
:stem
[latexmath]
++++
M =
\\begin{bmatrix}
-\\sin λ_0 & \\cos λ_0 & 0 \\\\
-\\sin φ_0 \\cos λ_0 & -\\sin φ_0 \\sin λ_0 & \\cos φ_0 \\\\
\\cos φ_0 \\cos λ_0 & \\cos φ_0 \\sin λ_0 & \\sin φ_0
\\end{bmatrix}
++++
The resulting XML will contain not LaTeX but MathML:
<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="M=\\begin{bmatrix}-\\sin\\lambda_{0}&\\cos\\lambda_{0}&0\\\\ -\\sin\\varphi_{0}\\cos\\lambda_{0}&-\\sin\\varphi_{0}\\sin\\lambda_{0}&\\cos\\varphi_{0% }\\\\ \\cos\\varphi_{0}\\cos\\lambda_{0}&\\cos\\varphi_{0}\\sin\\lambda_{0}&\\sin\\varphi_{0}% \\end{bmatrix}" display="block">
<mrow>
<mi>M</mi>
<mo>=</mo>
<mrow>
<mo>[</mo>
<mtable columnspacing="5pt" displaystyle="true" rowspacing="0pt">
<mtr>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mtd>
<mtd columnalign="center">
<mn>0</mn>
</mtd>
</mtr>
<mtr>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
<mo></mo>
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mo>-</mo>
<mrow>
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
<mo></mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
</mtd>
</mtr>
<mtr>
<mtd columnalign="center">
<mrow>
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
<mo></mo>
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mrow>
<mi>cos</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
<mo></mo>
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>λ</mi>
<mn>0</mn>
</msub>
</mrow>
</mrow>
</mtd>
<mtd columnalign="center">
<mrow>
<mi>sin</mi>
<mo></mo>
<msub>
<mi>φ</mi>
<mn>0</mn>
</msub>
</mrow>
</mtd>
</mtr>
</mtable>
<mo>]</mo>
</mrow>
</mrow>
</math>
@opoudjis so it will be metanorma --type std <path_to_file_with_ latexmath>
right?
@opoudjis should we change λ
to \lambda
in the input first?
No, we might as well use this to test the unicode2latex gem too.
@CAMOBAP795 Yes, and -t standoc -x xml would be sufficient
Or @opoudjis it might be better to have separate tests since they are two things.
Add documentation
Done in metanorma-standoc README.
OGC has a need to use latexmath in stem blocks. Since asciidoctor supports it I suppose we could too without much hassle.