`\minted@opt@quote` prevents expansion too much

muzimuzhi commented 1 year ago

Description and example

Before v2.7, the example below compiles without errors

\documentclass{article}
\usepackage{minted}

\begin{document}
\def\mylang{haskell}
\inputminted{\mylang}{code.hs}
\end{document}

Since v2.7, it fails with pygments error

Error: no lexer for alias '\\mylang ' found

because \minted@opt@quote{\mylang} prevents expansion of \mylang. See a report in #353.

Analysis The problem is introduced in https://github.com/gpoore/minted/commit/25e1f300159c3e7848026688415141508248d113. It changed

\MintedPygmentize\space -l #2
% to
\MintedPygmentize\space -l \minted@opt@quote{#2}

with \minted@opt@quote defined by https://github.com/gpoore/minted/blob/9ac85a3a1ae09936d80daf46927f55c51449555b/source/minted.dtx#L3003-L3007

Suggestion

\ifwindows
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter"\expanded{#1}"}}
\else
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter'\expanded{#1}'}}
\fi

gpoore commented 1 year ago

The \detokenize allows \minted@opt@quote to be used with things like escapeinside, which may involve punctuation/symbol characters that cannot be expanded (unless they are redefined to literal versions of themselves). Probably what is needed is putting the language inside a temp macro with \edef to expand.

muzimuzhi commented 1 year ago

The \detokenize allows \minted@opt@quote to be used with things like escapeinside, which may involve punctuation/symbol characters that cannot be expanded (unless they are redefined to literal versions of themselves).

Yes and no. \minted@opt@quote is only used in one place, in \minted@pygmentize to quote the language name. Things like escapeinside use another quoting macro, \minted@optlistcl@quote.

But yes \minted@opt@quote as is is rather good.

Probably what is needed is putting the language inside a temp macro with \edef to expand.

I agree with you that the language can be \edefed hence stored in a macro and I already find such one---\minted@lang, which is only set by \minted@configlang.

A new proposal:

In \minted@configlang, \edef the \minted@lang;
In \minted@pygmentize, use \minted@lang to replace its #2, and expand \minted@lang one-step before passing it to \minted@opt@quote;
Make all three \minted@pygmentize uses constant. For example, make them always be called with \minted@lang, not the value of \minted@lang. (Mostly because in \minted@pygmentize[#3]{\minted@lang} it's hard to pass the value of \minted@lang without either using newly added primitive \expanded (generally available in popular since texlive 2019) or adding an \edef.)
(optional) Now \minted@pygmentize doesn't make use of its #2 anymore. Simplification chance.

git diff

```diff diff --git a/source/minted.dtx b/source/minted.dtx index 6ff75d9..a7d67fb 100644 --- a/source/minted.dtx +++ b/source/minted.dtx @@ -2101,7 +2101,7 @@ % We need a way to check whether a language has had all its option list macros created. This generally occurs in a context where |\minted@lang| needs to be set. So we create a macro that does both at once. If the language list macros do not exist, we create them globally to simplify future operations. % \begin{macrocode} \newcommand{\minted@configlang}[1]{% - \def\minted@lang{#1}% + \edef\minted@lang{#1}% \ifcsname minted@optlistcl@lang\minted@lang\endcsname\else \expandafter\gdef\csname minted@optlistcl@lang\minted@lang\endcsname{}% \fi @@ -3022,7 +3022,8 @@ \detokenize{for /f "usebackq tokens=*"}\space\@percentchar\detokenize{a in (`kpsewhich}\space\minted@codefile\detokenize{`) do}\space \fi \fi - \MintedPygmentize\space -l \minted@opt@quote{#2} + \MintedPygmentize\space + -l \expandafter\minted@opt@quote\expandafter{\minted@lang} -f latex -P commandprefix=PYG -F tokenmerge \minted@optlistcl@g \csname minted@optlistcl@lang\minted@lang\endcsname \minted@optlistcl@inlines @@ -3318,7 +3319,7 @@ \minted@writecmdcode{#1}% \let\Verbatim\VerbEnv \let\endVerbatim\endVerbEnv - \expandafter\minted@pygmentize\expandafter{\minted@lang}% + \minted@pygmentize{\minted@lang}% \endgroup}} % \end{macrocode} % \end{macro} @@ -3405,7 +3406,7 @@ \begin{VerbatimOut}[codes={\catcode`\^^I=12},firstline,lastline]{\minted@jobname.pyg}}% {\end{VerbatimOut}% \minted@langlinenoson - \expandafter\minted@pygmentize\expandafter{\minted@lang}% + \minted@pygmentize{\minted@lang}% \minted@langlinenosoff}} % \end{macrocode} % \end{environment} @@ -3427,7 +3428,7 @@ \minted@configlang{#2}% \setkeys{minted@opt@cmd}{#1}% \minted@fvset - \minted@pygmentize[#3]{#2}% + \minted@pygmentize[#3]{\minted@lang}% \endgroup}} % \end{macrocode} % \end{macro} ```

drupol commented 1 year ago

Dear all,

I tried the proposed workaround:

\ifwindows
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter"\expanded{#1}"}}
\else
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter'\expanded{#1}'}}
\fi

When enabled, the original issue I had doesn't seems to show up, however another one related is coming (I don't know if it comes before the original issue).

ctfp-ocaml> Chapter 1.
ctfp-ocaml> (./content/1.1/category-the-essence-of-composition.tex [2]
ctfp-ocaml> ./content/1.1/category-the-essence-of-composition.tex:36: Use of \minted doesn'
ctfp-ocaml> t match its definition.
ctfp-ocaml> \\snip ...alse } \begin {mdframed}\begin {minted}[
ctfp-ocaml>                                                   #1]{#2}
ctfp-ocaml> l.36 \begin{snip}{text}
ctfp-ocaml> Output written on ctfp.xdv (13 pages, 140388 bytes).
ctfp-ocaml> Transcript written on ctfp.log.

I have a hunch that this is because of the snip environment which is defined as such:

\newenvironment{snip}[2][]
  {\VerbatimEnvironment
  \mdfsetup{
    linecolor=gray!20,
    linewidth=2pt,
    topline=false,
    bottomline=false,
    rightline=false
  }
  \begin{mdframed}\begin{minted}[#1]{#2}}
 {\end{minted}\end{mdframed}
 \vspace{-1ex}}

This environment has always been working before, maybe this is another issue?

Anyway, let me know if you need more information.

muzimuzhi commented 1 year ago

@drupol Those five lines need to be surrounded by \makeatletter ... \makeatother (for more explanation see https://tex.stackexchange.com/questions/8351/what-do-makeatletter-and-makeatother-do):

\makeatletter
\ifwindows
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter"\expanded{#1}"}}
\else
  \def\minted@opt@quote#1{\detokenize\expandafter{\expandafter'\expanded{#1}'}}
\fi
\makeatother

drupol commented 1 year ago

It works indeed!

Fixed in https://github.com/hmemcpy/milewski-ctfp-pdf/pull/306/commits/91779a5d1c3ac4f52e19dd4f1165fe0f4f2f8d95

gpoore commented 3 weeks ago

This is fixed in the minted v3.0 beta. \minted@opt@quote no longer exists. Settings are now expanded (\edef) before being passed to the Python side; there are no arbitrary invocations of \detokenize to interfere.

gpoore / minted

`\minted@opt@quote` prevents expansion too much #354