search

gpoore / minted

minted is a LaTeX package that provides syntax highlighting using the Pygments library. Highlighted source code can be customized using fancyvrb.
1.73k stars 125 forks source link

Trying to get `\inputminted` works with variable arguments #353

Closed drupol closed 1 year ago

drupol commented 1 year ago

Hello,

I'm working on refactoring the building of the open-source book at https://github.com/hmemcpy/milewski-ctfp-pdf Basically, it was using very old dependencies and I'm refreshing everything (see relevant PR https://github.com/hmemcpy/milewski-ctfp-pdf/pull/305)

After upgrading the LaTeX version and python version, I notice that I'm unable to build the document. I'm now stuck because \inputminted is not working as I would expect. Obviously, I might be wrong, but it seems that it doesn't use the value of a variable, but the variable itself. I spent the whole day yesterday trying to figure out what I was doing wrong, without luck.

I tried to reproduce a minimal working environment, here it is:

\documentclass{book}

\usepackage[cache=false,kpsewhich]{minted}

\def\OPTCustomLanguage{ocaml}
\def\OPTCustomLanguageExt{ml}

\NewDocumentCommand{\src}{m}{
  \srcsnippet{code/haskell/#1.hs}{haskell}
  \srcsnippet{code/\OPTCustomLanguage/#1.\OPTCustomLanguageExt}{\OPTCustomLanguage}
}
\NewDocumentCommand{\srcsnippet}{mm}{
  \inputminted{#2}{#1}
}
\begin{document}

\src{snippet01}

\end{document}

Create the src files:

touch code/haskell/snippet01.hs
touch code/ocaml/snippet01.ml

Then compile the document... and the error is:

(/nix/store/nr191iv6nas2ayhwg86v6rdnrymn5lmn-texlive-combined-2022/share/texmf/
tex/latex/l3backend/l3backend-xetex.def) (build/ctfp-ocaml.aux)
(/nix/store/nr191iv6nas2ayhwg86v6rdnrymn5lmn-texlive-combined-2022/share/texmf/
tex/latex/base/ts1cmr.fd) (./ctfp-ocaml.out.pyg) (./ctfp-ocaml.out.pyg)Error: no lexer for alias '\\OPTCustomLanguage ' found
system returned with code 256

Can you tell me if I'm doing something wrong here?

Thanks.

PS: after a bit of search on stackexchange, I found these related issues:

cc @hmemcpy

muzimuzhi commented 1 year ago

An expansion issue. Fully expanding args of \inputminted before it absorbs them does the trick.

\NewDocumentCommand{\srcsnippet}{mm}{
  \ExpandArgs{ee}
  \inputminted{#2}{#1}
}

Note: \ExpandArgs is added in LaTeX2e 2022-06-01 (see texdoc ltnews), for prior LaTeX releases you can just use \expanded{\noexpand\inputminted{...}{...}}.

drupol commented 1 year ago

Wow nice. Is it documented somethere ?

muzimuzhi commented 1 year ago

Don't think so. (I don't think it's documented.)

I need some more time to dig into the real culprit and then I will come back with suggestion about which minted public and/or internal commands could fully expand their selected arguments to ease the problem like this one.

drupol commented 1 year ago

Ok.

Where can I find documentation about ExpandArgs ? I'd like to understand the meaning of ee.

muzimuzhi commented 1 year ago

Where can I find documentation about ExpandArgs ? I'd like to understand the meaning of ee.

Ah apart from mentioned in the LaTeX News (run texdoc ltnews to open the pdf), it's documented in the new LaTeX user guide, sec. 4. Just run texdoc usrguide to open the guide in pdf.

If you don't have a local LaTeX distribution installation or don't have docs installed, go to https://texdoc.org and search for usrguide.

drupol commented 1 year ago

I found the documentation about \ExpandArgs at https://www.latex-project.org/help/documentation/usrguide3.pdf but it's not exhaustive.

Last things,

When I tried in my initial example, it worked.

However, when I try to do in my project where everything is a bit more complex:

\documentclass{book}

\usepackage[cache=false,kpsewhich]{minted}

\def\OPTCustomLanguage{ocaml}
\def\OPTCustomLanguageExt{ml}

\NewDocumentCommand\src{mO{}}{
  \srcsnippet{code/haskell/#1.hs}{blue}{haskell}{#2}
  \ifdefined\OPTCustomLanguage{%
    \unskip
    \srcsnippet{code/\OPTCustomLanguage/#1.\OPTCustomLanguageExt}{\OPTCustomLanguageColor}{\OPTCustomLanguage}{#2}
  }
  \fi
  \NoIndentAfterThis
}

\NewDocumentCommand\srcsnippet{mmmm}{
  \ExpandArgs{ee}
  \newif\ifbreak
  \mdfsetup{%
    linecolor=#2!20,%
    linewidth=2pt,
    topline=false,
    bottomline=false,
    rightline=false,
  }
  \def\x{#4}
  \ifthenelse{\equal{\x}{b}}{\breaktrue}{\breakfalse}
  \begin{mdframed}
    \ifbreak \inputminted[autogobble,breaklines]{#3}{#1}
    \else \inputminted[autogobble]{#3}{#1}
    \fi
  \end{mdframed}
}

\begin{document}

\src{snippet01}

\end{document}

Can you help me on this as well ? I honestly do not know where to add \ExpandArgs and what to put inside.

muzimuzhi commented 1 year ago

Now it works. Notable changes

\documentclass{book}
\usepackage{mdframed}
\usepackage[cache=false,kpsewhich]{minted}
\usepackage{xcolor}

\def\OPTCustomLanguage{ocaml}
\def\OPTCustomLanguageExt{ml}
\def\OPTCustomLanguageColor{orange}

\NewDocumentCommand\src{mO{}}{
  \srcsnippet{code/haskell/#1.hs}{blue}{haskell}{#2}
  \ifdefined\OPTCustomLanguage{%
    \unskip
    \expanded{%
      \srcsnippet{code/\OPTCustomLanguage/#1.\OPTCustomLanguageExt}{\OPTCustomLanguageColor}{\OPTCustomLanguage}{#2}%
    }%
  }%
  \fi
  % \NoIndentAfterThis
}

\newif\ifbreak

\NewDocumentCommand\srcsnippet{mmmm}{
  \mdfsetup{%
    linecolor=#2!20,%
    linewidth=2pt,
    topline=false,
    bottomline=false,
    rightline=false,
  }
  \def\x{#4}
  \ifthenelse{\equal{\x}{b}}{\breaktrue}{\breakfalse}
  \begin{mdframed}
    \ifbreak \inputminted[autogobble,breaklines]{#3}{#1}
    \else    \inputminted[autogobble]{#3}{#1}
    \fi
  \end{mdframed}
}

\begin{document}

\src{snippet01}

\end{document}
drupol commented 1 year ago

Indeed, adding \expanded worked immediately. I don't know how to thank you !!!

I think this is the hardest issue I ever encountered with LaTeX so far.

Thanks !!!

muzimuzhi commented 1 year ago

I think this definition of \srcsnippet is better, see below.

It won't pollute global mdframed settings and doesn't need the conditional \ifbreak. (I forgot to mention that \newif\ifbreak was moved outside \srcsnippet in my previous example.) Therefore you can get rid of the extra group of braces in \src, \ifdefined\OPTCustomLanguage{...}\fi.

\NewDocumentCommand\srcsnippet{mmmm}{
  \begin{mdframed}[
    linecolor=#2!20,
    linewidth=2pt,
    topline=false,
    bottomline=false,
    rightline=false]
    % \ifstrequal is defined by etoolbox package, which minted already loads
    \expanded{%
      \ifstrequal{#4}{b}
        {\noexpand\inputminted[autogobble,breaklines]{#3}{#1}}
        {\noexpand\inputminted[autogobble]{#3}{#1}}%
    }
  \end{mdframed}
}
drupol commented 1 year ago

I'll try that and report back here. Thanks again !!!

That said, the ifdefined in \src is necessary in case if the language is not defined (which can happen). Are you sure I can get rid of it ?

muzimuzhi commented 1 year ago

I need some more time to dig into the real culprit and then I will come back with suggestion about which minted public and/or internal commands could fully expand their selected arguments to ease the problem like this one.

It's a bug introduced in minted v2.7, see #354 and welcome to test the suggestion in it.

unalmis commented 1 year ago 
\usepackage{minted}
\usepackage{subfiles}
% first argument is section name, second is file name, third is code language
\NewDocumentCommand{\codesection}{mmm}{%
    \addcontentsline{toc}{subsection}{Code: #1}
    \inputminted{#3}{\subfix{#2}}
}

After upgrading from texlive 2021 to texlive 2022, my \codesection command no longer works (1000 errors on missing $ inserted, mispaced alignment tab character etc). Removing the subfix command solves the issue, although I lose its feature. The \subfix command works otherwise, so it might be an incompatibility between minted and subfiles introduced by the bug discussed in here. @muzimuzhi is this another expansion issue with minted? If so, do you know of any workaround? The output of tlmgr info minted gives cat version 2.6 and and for subfiles package gives 2.2.

If I rollback to an older version by using \usepackage{minted1}, then the issue goes away.

muzimuzhi commented 1 year ago

@unalmis I think it's the same problem (the filename of \inputminted is not expanded since minted v2.7) and the workarounds I previously suggested should work for your case as well. If none of them works you can prepare a complete example and file a new issue.

Another idea making use of argument processors (see texdoc usrguide, sec. 2.10 Argument processors):

\usepackage{minted}
\usepackage{subfiles}
% first argument is section name, second is file name, third is code language
\NewDocumentCommand{\codesection}{ m m >{\wrapInSubfix} m }{%
    \addcontentsline{toc}{subsection}{Code: #1}%
    \inputminted{#3}{#2}%
}

% or \NewExpandableDocumentCommand{\wrapInSubfix}{m}{...} if you prefer it
\newcommand{\wrapInSubfix}[1]{\edef\ProcessedArgument{\subfix{#1}}}
unalmis commented 1 year ago

Thanks for the attempted solution. Unfortunately my issue persisted. I may open an issue much later with a working example. For now I have found an alternative that works for me. sharing in case others find it useful:

\usepackage[dvipsnames]{xcolor}
\usepackage{listings}
\usepackage{subfiles}

% code listings
\definecolor{codebackground}{gray}{0.973}  % f8f8f8
\lstset{
    basicstyle={\ttfamily \footnotesize},
    breaklines=true,
    columns=fullflexible,
    emptylines=1,
    frame=single,
    frameround=tttt,
    keepspaces=true,
    language=Python,
    numberblanklines=false,
    numbers=left,
    numberstyle=\tiny,
    showstringspaces=false,
    upquote=true,
    % syntax highlighting
    backgroundcolor=\color{codebackground},
    commentstyle=\color{Bittersweet},
    keywordstyle=\color{MidnightBlue},
    stringstyle=\color{Bittersweet},
}
\RenewDocumentCommand{\lstlistingname}{}{Code}
\RenewDocumentCommand{\lstlistlistingname}{}{Codes}
% easier to remember command name
\NewDocumentCommand{\codelisting}{mm}{%
    % first argument is file name
    \lstinputlisting[caption=#2]{\subfix{#1}}
}