gpoore / minted

minted is a LaTeX package that provides syntax highlighting using the Pygments library. Highlighted source code can be customized using fancyvrb.
1.73k stars 125 forks source link

[bug][compatibility] `Verbatim` environment generated by `minted` with `mathescape` broken with `babel`'s `magyar.ldf` (Hungarian 🇭🇺 language) #382

Open whisperity opened 6 months ago

whisperity commented 6 months ago

According to the documentation referenced in the babel-hungarian package, available at http://math.bme.hu/latex/magyarldf-doc.pdf § 2.6 (page 10), this package makes ` (backtick) an "active character" which is equivalent to Babel's "Shorthand" terminology (http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf, § 1.10, page 11), supposedly required for getting Babel to render special characters. As a quick translation for others not speaking Hungarian (and magyar.ldf has no English documentation...), the documentation shows use cases such as typing `< and `> to have TeX render « and ».

These things are absolutely unnecessary in the modern world with csquotes and UTF-8 input, but the problem is: using babel-hungarian together with minted results in the document not building.

I have tried various other Babel languages (Czech, French, German, Russian, Spanish) and it looks like none other has this problem.

I found that not using mathescape fixes the issue (the codes={\catcode... part is not generated into the .pygtex files), but this is not possible for cases where mathescape is needed for the document.

Luckily, I found that disabling magyar.ldf's "active character" behaviour also fixes the problem.

\PassOptionsToPackage{active=onlycs}{magyar.ldf}

The documentation for magyar.ldf says that

"If we type \string` instead of `, the meaning of the \"active character\" disappears. Thus, \string`\string` is an English opening quote, assuming the current font has such a ligature."

Could you please code minted to use this \string` instead of ` in the generated code, or at least show a warning if it detects that babel-hungarian is loaded and used?

This would save a LOT of headache for other users... this took me a long time to figure out, and even now I don't fully understand what this "shorthand" stuff is for.

MWE doc.tex

Install babel-hungarian.

\documentclass{beamer}

% \PassOptionsToPackage{active=onlycs}{magyar.ldf}

\usepackage[magyar]{babel}
\usepackage{minted}

\begin{document}

\begin{frame}[fragile]{Hello, World!}
  \begin{minted}[fontsize=\small,mathescape]{CPP}
    int main() {
      return 2;
    }
  \end{minted}
\end{frame}

\end{document}

Compile using latexmk -xelatex -shell-escape doc or latexmk -lualatex -shell-escape doc.

Version info

❯ tlmgr info beamer babel babel-hungarian minted | grep "^package\|^cat-version"
package:     beamer
cat-version: 3.70
package:     babel
cat-version: 3.95
package:     babel-hungarian
cat-version: 1.5c
package:     minted
cat-version: 2.8

Observed error

The file generated by minted:

\begin{Verbatim}[commandchars=\\\{\},codes={\catcode`\$=3\catcode`\^=7\catcode`\_=8\relax}]
\PYG{+w}{    }\PYG{k+kt}{int}\PYG{+w}{ }\PYG{n+nf}{main}\PYG{p}{()}\PYG{+w}{ }\PYG{p}{\PYGZob{}}
\PYG{+w}{      }\PYG{k}{return}\PYG{+w}{ }\PYG{l+m+mi}{2}\PYG{p}{;}
\PYG{+w}{    }\PYG{p}{\PYGZcb{}}
\end{Verbatim}

With XeLaTeX

(./presentation.nav) (./presentation.vrb
(./_minted-presentation/default.pygstyle)
(./_minted-presentation/11BD279AAA8F1FFA80A1B7D2945FD51B3E174A47E35DB6814BABFE5
4A3627AC3.pygtex
! Missing number, treated as zero.
<to be read again>
                   \XeTeXglyph
l.1 ...code`\$=3\catcode`\^=7\catcode`\_=8\relax}]

With LuaLaTeX

(./presentation.nav) (./presentation.vrb
(./_minted-presentation/default.pygstyle)
(./_minted-presentation/11BD279AAA8F1FFA80A1B7D2945FD51B3E174A47E35DB6814BABFE5
4A3627AC3.pygtex
! Missing number, treated as zero.
<to be read again>
\@remove@tlig@@@
l.1 ...atcode`\$=3\catcode`\^=7\catcode`\_=8\relax}]
muzimuzhi commented 6 months ago

Luckily, I found that disabling magyar.ldf's "active character" behaviour also fixes the problem.

\PassOptionsToPackage{active=onlycs}{magyar.ldf}

You can also use \shorthandoff{`}.

The file generated by minted:

\begin{Verbatim}[commandchars=\\\{\},codes={\catcode`\$=3\catcode`\^=7\catcode`\_=8\relax}]
\PYG{+w}{    }\PYG{k+kt}{int}\PYG{+w}{ }\PYG{n+nf}{main}\PYG{p}{()}\PYG{+w}{ }\PYG{p}{\PYGZob{}}
\PYG{+w}{      }\PYG{k}{return}\PYG{+w}{ }\PYG{l+m+mi}{2}\PYG{p}{;}
\PYG{+w}{    }\PYG{p}{\PYGZcb{}}
\end{Verbatim}

The codes={\catcode`\$=3\catcode`\^=7\catcode`\_=8\relax} is generated by pygments python library (hence not under control of minted, see https://github.com/pygments/pygments/blob/b40ee2f0f64223e15984a748d7cbcf9cd25ffeeb/pygments/formatters/latex.py#L352-L354.

But this doesn't mean the problem cannot be fixed on minted's side. Similar to #158, minted can reset catcode of backtick (`) just before loading pygment output.

\documentclass{beamer}
\usepackage[magyar]{babel}
\usepackage{minted}

\usepackage{xpatch}

\makeatletter
\xpretocmd\minted@input
  {\catcode``=12 }
  {}{\PatchFailed}
\makeatother

\begin{document}

\begin{frame}[fragile]{Hello, World!}
  \begin{minted}[fontsize=\small,mathescape]{CPP}
    int main() {
      return 2;
    }
  \end{minted}
\end{frame}

\end{document}