`inputminted` quoted arguments for `<langage>` differ from `xelatex`

goyalyashpal commented 11 months ago

details

pdflatex treats the unquoted psql ... as a whole as the alias name
tectonic interprets the quoted "psql ..." as pygmentize -l "psql -F keywordcase:case=upper" -f latex ...

the syntax for command is: \inputminted[⟨options⟩]{⟨language⟩}{⟨filename⟩}

it seems that:

minted from tectonic is passing the argument directly without any manipulation
minted from pdflatex(ctan) is internally enclosing the argument for {<language>} in quotes, and so, on using a quoted argument, the quotes matches up and the supplied argument essentially becomes naked one.
i.e. like this: pygmentize -l "<language>" ... which with quoted \ becomes: pygmentize -l ""<language>"" ...

discussion

is this difference reproducible on ur system?
is it known and intended?
is it documented anywhere?

logs

Console logs _(click to expand)_

`tectonic` on quoted `"psql ..."` ```console $ tectonic -X build 2>&1 | grep -EniA1 "psql" 24:note: running shell command: `pygmentize -l "psql -F keywordcase:case=upper" -f latex -P commandprefix=PYG -F tokenmerge -P stripnl=False -o build/auxx/_minted-default/44770DE0266CD260D51BC400B74EE485704DABC3C20AA2C88B60138D5195D5EB.pygtex ../archive/lib/test.mysqlcon` 25:Error: no lexer for alias '"psql' found 26-warning: command exited with error code 1 27:error: failed to execute the shell-escape command "pygmentize -l "psql -F keywordcase:case=upper" -f latex -P commandprefix=PYG -F tokenmerge -P stripnl=F alse -o build/auxx/_minted-default/44770DE0266CD260D51BC400B74EE485704DABC3C20AA2C88B60138D5195D5EB.pygtex ../archive/lib/test.mysqlcon": execution of the re quest failed 28-error: ./tex/definitions.tex:30: Package minted Error: Missing Pygments output; \inputminted was ``` `pdflatex` on unquoted "psql ..." ```console $ pdflatex.exe --shell-escape -output-directory=.\build -aux-directory=.\build/auxx -synctex=1 -interaction=nonstopmode "document-5th-dbmsl-mysql".tex | grep -Eni "psql" Error: no lexer for alias 'psql -F key/wordcase:case=upper' found 125:l.33 ...utmintedpsql{../archive/lib/test.mysqlcon} ```

Neved4 commented 11 months ago

Hey @goyalyashpal thx for the issue! 🩵

Could you provide a MWE for me to test things out? Also could you share the output when using xelatex?

goyalyashpal commented 11 months ago

provide a MWE share the output when using xelatex

sure! here's full reproduction details:

FILES TREE:

% ./src/document.tex
\begin{filecontents}[noheader]{./test.psql}
    mysql-> select * from thistable;
\end{filecontents}

\documentclass{article}
\usepackage{minted}

\begin{document}
% \inputminted{psql}{test.psql}
% \inputminted{psql -F keywordcase:case=upper}{test.psql}    % <- Naked: works in tectonic
\inputminted{"psql -F keywordcase:case=upper"}{test.psql}    % <- "Quoted": works in {pdf,xe}latex
\end{document}

Other / Trivial files

```console $ tree -aC --dirsfirst --noreport . ├── src │ ├── _postamble.tex │ ├── _preamble.tex │ └── document.tex ├── .latexmkrc └── Tectonic.toml ``` ```toml # ./Tectonic.toml [doc] name = "latexmk_texlab" bundle = "https://data1.fullyjustified.net/tlextras-2022.0r0.tar" [[output]] name = "default" type = "pdf" shell_escape = true index = "document.tex" ``` ```perl # ./.latexmkrc $pdf_mode = 1; # tex -> pdf # $pdf_mode = 5; # tex -> xdv -> pdf @default_files = ('src/document.tex'); # $aux_dir='_build/auxx'; # AVOID .: complications with filecontents # $out_dir='_build'; # AVOID .: complications with filecontents ```

### ENVIRONMENT & VERSIONS:

```console $ systeminfo | grep -iA1 "os name" OS Name: Microsoft Windows 10 Home Single Language OS Version: 10.0.19045 N/A Build 19045 $ tectonic -V Tectonic 0.14.1 $ pdflatex --version | head -n1 - MiKTeX-pdfTeX 4.16 (MiKTeX 23.10.12) $ xelatex --version | head -n1 - MiKTeX-XeTeX 4.10 (MiKTeX 23.10.12) ```

### CONSOLE I/O:

*tex compatible ("quoted")

```console $ pdflatex --shell-escape -interaction=nonstopmode src/document.tex | tail -n4 - /cm/cmr12.pfb> Output written on document.pdf (1 page, 17232 bytes). Transcript written on document.log. $ xelatex --shell-escape -interaction=nonstopmode src/document.tex | tail -n4 - A7B.pygtex) [1] (document.aux) ) (see the transcript file for additional information) Output written on document.pdf (1 page). Transcript written on document.log. $ tectonic -X build 2>&1 | grep -niA9 "psql" 23:note: running shell command: `pygmentize -l "psql -F keywordcase:case=upper" -f latex -P commandprefix=PYG -F tokenmerge -P stripnl=False -o _minted-default/C08E6104B7DE77ADBE04D2869C2F686CB43FCAD6DFDF36838D64F64DE4C714A2.pygtex test.psql` 24:Error: no lexer for alias '"psql' found 25-warning: command exited with error code 1 26:error: failed to execute the shell-escape command "pygmentize -l "psql -F keywordcase:case=upper" -f latex -P commandprefix=PYG -F tokenmerge -P stripnl=False -o _minted-default/C08E6104B7DE77ADBE04D2869C2F686CB43FCAD6DFDF36838D64F64DE4C714A2.pygtex test.psql": execution of the request failed 27-error: document.tex:19: Package minted Error: Missing Pygments output; \inputminted was 28-probably given a file that does not exist--otherwise, you may need 29-the outputdir package option, or may be using an incompatible build tool, 30-or may be using frozencache with a missing file. 31- 32-See the minted package documentation for explanation. 33-Type H for immediate help 34-error: halted on potentially-recoverable error as specified ```

tectonic compatible (naked)

```console $ tectonic -X build 2>&1| tail -n4 - warning: warnings were issued by the TeX engine; use --print and/or --keep-logs for details. note: Running xdvipdfmx ... note: Writing `.\build\default\default.pdf` (6.78 KiB) note: Skipped writing 2 intermediate files (use --keep-intermediates to keep them) $ xelatex --shell-escape -interaction=nonstopmode src/document.tex 1>/dev/null Error: no lexer for alias 'psql -F keywordcase:case=upper' found $ pdflatex --shell-escape -interaction=nonstopmode src/document.tex 1>/dev/null Error: no lexer for alias 'psql -F keywordcase:case=upper' found ```

Neved4 commented 11 months ago

So far I can't reproduce on macOS/Linux, maybe @vlasakm has some ideas;)

goyalyashpal commented 11 months ago

i.e. for you, tectonic was working on "Quoted" one, and not on naked one; right?

Neved4 commented 11 months ago

@goyalyashpal so far only \inputminted{psql}{test.psql} compiles, and both \inputminted{psql -F keywordcase:case=upper}{test.psql} and \inputminted{"psql -F keywordcase:case=upper"}{test.psql} seem to fail.

vlasakm commented 11 months ago

Thanks for the MWE, it got me started.

I think the problem is your usage of \inputminted. According to the documentation, the syntax is as follows:

\inputminted[⟨options⟩]{⟨language⟩}{⟨filename⟩}

So the first mandatory argument is the language. The fact that it gets literally input into the command line invocation of pygmentize is just a side effect, you shouldn't be passing any other options there. The minted package is striving to provide a higher level interface, and it is best to not sidestep it.

But, you want -F keywordcase:case=upper as part of the command line. So I searched for -F in the documentation, and found this:

\minted@def@optcl{keywordcase}{-F keywordcase:case}{#1}

It's from the implementation part, and the details are not important, but it essentially tells us that the keywordcase option is available, and sure enough it is actually documented (in the user part of the documentation) and can take upper as an argument. To use it, we would do:

\inputminted[keywordcase=upper]{psql}{test.psql}

Full MWE:

\begin{filecontents}[noheader]{./test.psql}
    mysql-> select * from thistable;
\end{filecontents}

\documentclass{article}
\usepackage{minted}

\begin{document}

\inputminted[keywordcase=upper]{psql}{test.psql}

\end{document}

In any case the quoted use is wrong, -F keywordcase:case=upper would be regarded as a single argument by the shell, while two arguments are desired (-F and keywordcase:case=upper).

Now, it is actually interesting that the following works:

\begin{minted}{psql -F keywordcase:case=upper}
    mysql-> select * from thistable;
\end{minted}

While this does not (neither quoted nor bare) as also reproduced by @Neved4 :

\begin{filecontents}[noheader]{./test.psql}
    mysql-> select * from thistable;
\end{filecontents}

\inputminted{psql -F keywordcase:case=upper}{test.psql}

In the first case minted creates a temporary file itself and the paths work out correctly. In the second case the file resides in the same directory as the input file and under the current rules cannot be found by any program ran with -Z shell-escape. For this -Z shell-escape-cwd=... must be used. For example I compiled with tectonic -Z shell-escape -Z shell-escape-cwd=$(pwd) test-1114.tex to run programs executed by Tectonic in the same directory as the test.psql file.

The help message actually mentions this exact use case:

$ tectonic -Z help test-1114.tex
Available unstable options:

[...]
    -Z shell-escape             Enable \write18
    -Z shell-escape-cwd         Working directory to use for \write18. Use $(pwd) for same behaviour as
                                    most other engines (e.g. for relative paths in \inputminted).
                                    Implies -Z shell-escape
[...]

goyalyashpal commented 11 months ago

While this does not (neither quoted nor bare)

i wonder why it worked for me. anyhow, thanks a lot for such digging in the minted documentation.

and also, what's the conclusion? should this behave same as {xe,pdf}latex or how?

vlasakm commented 11 months ago

While this does not (neither quoted nor bare)

i wonder why it worked for me. anyhow, thanks a lot for such digging in the minted documentation.

It very likely worked for you because you are Windows, whereas I tested only on Linux.

and also, what's the conclusion? should this behave same as {xe,pdf}latex or how?

For me personally, anything that doesn't behave exactly as xelatex is a bug (minus of course the intended differences, like here the need to set -Z shell-escape-cwd). Here, the behaviour actually differs (at least on Linux) in that the bare variant works in Tectonic, and doesn't in xelatex. IMO we should check why Tectonic behaves differently in this case, and get the same behavior.

goyalyashpal commented 11 months ago

IMO we should check why Tectonic behaves differently in this case, and get the same behavior

all the surface level specifics are figured out.
this personally don't bother me either - with the builtin options feature from minted you shared.

so, i leave up to you (and others) to decide when to close this issue 😃 thanks a lot for all the work and diagnostics 😇

vlasakm commented 11 months ago

You uncovered the different behavior and that needs fixing. I suggest to keep the issue open, until we decide whether to do something about it. We could probably start a new issue, but the context is already here.

tectonic-typesetting / tectonic