[ltcmd] Verbatim command breaks with backslash as a delimiter

[PhelypeOleinik]

Brief outline of the bug

Spotted here, a command defined with a v argument breaks when a catcode 0 character is used as delimiter for the argument, as in \verb\foo\ (which is valid for the standard LaTeX \verb).

Either the diagnostics should be fixed to throw an appropriate error about \ not being possible as a delimiter, or the scanner should be fixed to allow \ as \verb does.

Minimal example showing the bug

\RequirePackage{latexbug}
\documentclass{article}
\NewDocumentCommand{\myverb}{v}{\texttt{#1}}
\def\foo{wrong}
\begin{document}
\verb\foo\

\myverb\foo\
\end{document}

Log file (required) and possibly PDF file

test.log

Explanation for the current behaviour

The v scanner starts (\__cmd_grab_v_aux:w) by using \peek_remove_spaces:n, which uses \futurelet in \foo, freezing it as a control sequence, then checks if the next token is a {, which is false, and then checks if the next token is an N-type, which is true. At this point, the next token should be a character, so \__cmd_grab_v_aux_test:N checks \foo for that, which is not, thus calling \__cmd_grab_v_aux_abort:n to raise an error. \__cmd_grab_v_aux_abort:n then assumes that, since it saw a non-character token, that the verbatim command was misused in an argument.

[FrankMittelbach]

\verb does not only accept \ it also accepts %, {, or } all of which make very weird input and all of which give strange diagnostics with "v"

\documentclass{article}

\NewDocumentCommand\baz{v}{\texttt{#1}}

\begin{document}

\typeout{All working}

\verb\foo\

\verb{foo{

\verb}foo}

\verb%foo%

\typeout{Not working}

\baz%foo%

\baz}foo}

\baz{foo{

\baz\foo\

\end{document}

I'm tempted to consider the \verb behavior unsupported and and even disallow it going forward (even if that may break a very small number of documents). Nevertheless the diagnostics given by \NewDocumentCommand are something we should fix for all cases.

[car222222]

I tend to agree that use of these should be deprecated with \verb etc. : \ { } (catcode < 3 ).

^ (catcode 7) is I think (like %, catcode ) a bit tricky to document its behaviour, due to ^^ (etc)-notation; so perhaps these two should join the list of unwise choices.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity.

[josephwright]

@PhelypeOleinik You still happy to have this one on your to-do list?

[car222222]

The decision on what to do (probably in detail) should be on all our lists!

[PhelypeOleinik]

@josephwright yeah, no problem. This one isn't too complicated once we decide exactly what to do. I agree with Frank, that \ and % are weird choices for delimiters. { is documented to be supported (if memory serves me right), but then closed by }, so that should probably stay. The other two should get appropriate errors.

I'm not sure about ^ as Chris mentioned. It is special when it is catcode 7, but in verbatim it is made catcode 12, so it's an ordinary character (same as \ and %, but...)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.

[car222222]

This was never completed only because we did not make any decisions concerning which characters to allow

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.

[car222222]

See previous explanation for the continuing lack of action.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity.