imgmath crashes if the equations contains Japanese text

tk0miya commented 7 years ago

Problem

imgmath crashes if the formula contains Japanese text

Procedure to reproduce the problem

.. math::

   E = mc^2 \mbox{ハロー}

Even if I added imgmath_latex = 'platex' to my conf.py, it is also failed. Because dvipng does not support Japanese text...

Error logs / results

WARNING: inline latex u'E = mc^2 \\mbox{\u30cf\u30ed\u30fc}': latex exited with error
[stdout]
This is pdfTeX, Version 3.14159265-2.6-1.40.17 (TeX Live 2016) (preloaded format=latex)
 restricted \write18 enabled.
entering extended mode
(./math.tex
LaTeX2e <2016/03/31>
Babel <3.9r> and hyphenation patterns for 83 language(s) loaded.
(/usr/local/texlive/2016/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2016/texmf-dist/tex/latex/base/size12.clo))
(/usr/local/texlive/2016/texmf-dist/tex/latex/base/inputenc.sty
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/utf8x.def))
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/ucs.sty
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uni-global.def))
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsmath/amsmath.sty
For additional information on amsmath, use the `?' option.
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsmath/amstext.sty
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsmath/amsgen.sty))
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsmath/amsbsy.sty)
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsmath/amsopn.sty))
(/usr/local/texlive/2016/texmf-dist/tex/latex/amscls/amsthm.sty)
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsfonts/amssymb.sty
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsfonts/amsfonts.sty))
(/usr/local/texlive/2016/texmf-dist/tex/latex/anyfontsize/anyfontsize.sty)
(/usr/local/texlive/2016/texmf-dist/tex/latex/tools/bm.sty) (./math.aux)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/ucsencs.def)
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsfonts/umsa.fd)
(/usr/local/texlive/2016/texmf-dist/tex/latex/amsfonts/umsb.fd)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uni-48.def)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uninames.dat)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uni-48.def)

! Package ucs Error: Unknown Unicode character 12495 = U+30CF,
(ucs)                possibly declared in uni-48.def.
(ucs)                Type H to see if it is available with options.

See the ucs package documentation for explanation.
Type  H <return>  for immediate help.
 ...

l.14 ...split}E = mc^2 \mbox{ハロー}\end{split}

(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uninames.dat)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uni-48.def)

! Package ucs Error: Unknown Unicode character 12525 = U+30ED,
(ucs)                possibly declared in uni-48.def.
(ucs)                Type H to see if it is available with options.

See the ucs package documentation for explanation.
Type  H <return>  for immediate help.
 ...

l.14 ...split}E = mc^2 \mbox{ハロー}\end{split}

(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uninames.dat)
(/usr/local/texlive/2016/texmf-dist/tex/latex/ucs/data/uni-48.def)

! Package ucs Error: Unknown Unicode character 12540 = U+30FC,
(ucs)                possibly declared in uni-48.def.
(ucs)                Type H to see if it is available with options.

See the ucs package documentation for explanation.
Type  H <return>  for immediate help.
 ...

l.14 ...split}E = mc^2 \mbox{ハロー}\end{split}

Package ucs Warning: Unknown character 12495 = 0x30CF appeared again. on input
line 14.

Package ucs Warning: Unknown character 12525 = 0x30ED appeared again. on input
line 14.

Package ucs Warning: Unknown character 12540 = 0x30FC appeared again. on input
line 14.

[1] (./math.aux) )
(see the transcript file for additional information)
Output written on math.dvi (1 page, 416 bytes).
Transcript written on math.log.

Expected results

Able to build the equations

tk0miya commented 7 years ago

As a workaround, I prepared simple script named dvipngja. It uses dvips and imagemagick to make images.

$ cat dvipngja
#!/bin/sh

set -ex

while getopts o:T:z: OPT
do
    case $OPT in
      "o") OUTPUT="$OPTARG";;
    esac
done
shift `expr $OPTIND - 1`

PSFILE=`mktemp`
dvips "$1" -o "$PSFILE"
convert -trim "$PSFILE" "$OUTPUT"
rm $PSFILE

$ tail -4 conf.py
imgmath_latex = 'platex'
imgmath_dvipng = '/Users/tkomiya/work/tmp/doc/dvipngja'
imgmath_dvipng_args = []
imgmath_font_size = 24

chezou commented 7 years ago

I found a workaround. Using svg with setting font map works fine for me.

Example conf.py:

imgmath_latex = 'platex'
imgmath_image_format = 'svg'
imgmath_dvisvgm_args = ["-m=ptex-ipa.map"]
imgmath_font_size = 22

jfbu commented 7 years ago

Sorry for overlooking this issue for some time. It appears it works with

.. math::

   E = mc^2 \text{ハロー}

image: capture d ecran 2017-05-05 a 20 39 11

can you try it ?

jfbu commented 7 years ago

oh sorry actually it also works for me with your original example with \mbox!! I tested this on Sphinx 1.5.5 with TeXLive 2016 in its final version. (TL2017 will be released soon, and TL2016 is frozen forever).

From your error log it appears you were using utf8x option not utf8, because package ucs is loaded, which is what utf8x does. Where does the utf8x come from ?

tk0miya commented 7 years ago

Really? At my local, both \text and \mbox are failed. AFAIK, dvipng does not support Japanese text. So this is not only the problem of LaTeX notation.

From your error log it appears you were using utf8x option not utf8, because package ucs is loaded, which is what utf8x does. Where does the utf8x come from ?

sphinx.ext.imgmath uses utf8x option yet: https://github.com/sphinx-doc/sphinx/blob/1.5.5/sphinx/ext/imgmath.py#L46.

jfbu commented 7 years ago

Hi @tk0miya I am just stupid. I, of course, only tested with make latexpdf. LaTeX is my obsession ! Hence I simoly did not realize your problem is with make html although you mentioned very clearly dvipng. Some days I feel I would need a way to inject directly some dopamine in the brain. Of course now I confirm your problem. I will investigate when I get time, which is probably not today...

jfbu commented 7 years ago

for png output:

as an alternative to your dvipngja, I checked that the following work:

arrange so that DOC_HEAD has \documentclass[12pt]{jreport} and \usepackage[T1]{fontenc} and that it does not use inputenc.
set imgmath_latex = 'platex'
the called script should execute:
- dvipdfmx, (produces math.pdf)
- pdfcrop, (produces math-crop.pdf)
- gs -dSAFER -dNOPAUSE -dBATCH -sDEVICE=pngalpha -r450 -dDownScaleFactor=3 -sOutputFile=<outputfile.png> -- math-crop.pdf

I use the last incantation when my goal is 150dpi, like the -D150 option of dvipng. (the 150 is thus obtained as 450 divided by 3). I see that the default in imgmath.py is 110, so perhaps rather for example -r330 -dDownScaleFactor=3 (edit or 440 divided by 4). The -pngalpha gives transparent background gs doc.

I inserted directly suitable usage of Popen in imgmath.py to do the above commands, just as a proof of concept and it works with make html.

Unrelated note: your dvipngja uses convert but I think on Windows, convert is something completely different from the ImageMagick one. (the windows convert is related to formatting hard disks I think)

tk0miya commented 7 years ago

Thanks, it looks better than my ad-hoc way! I will consider how to embed it into sphinx.ext.imgmath later.

jfbu commented 7 years ago

I said jreport because Sphinx uses it for main document, but jarticle would be ok to. Perhaps there is even a way with less overhead (minimal class ? but should check if fine with loading the ams packages, and bm package)

sphinx-doc / sphinx