Non-standard command line parsing misinterprets semicolons inside arguments

michkot commented 2 years ago

Windows Terminal version

1.13.11431.0

Windows build number

10.0.19044.1706

Other Software

MSYS2 / Git for Windows bash (but really anything that regularly uses semicolons in their arguments, including filepaths)

Steps to reproduce

Relates to

https://github.com/microsoft/terminal/pull/11314#issuecomment-926399451 https://github.com/microsoft/terminal/pull/11314#issuecomment-926742571 I hope that given @lhecker already expressed his doubts about the current behaviour, there would be some motivation for a fix-by-breaking-change here.

Use your favorite way to create a process on windows, I used

using System;

namespace CreateProcess
{
   class Program
   {
      static void Main(string[] args)
      {

         while (true)
         {
            var line = System.Console.ReadLine();
            try
            {
               var command = line.Substring(0, line.IndexOf(" "));
               var args2 = line.Substring(line.IndexOf(" ") + 1);
               System.Diagnostics.Process.Start(command, args2);
            } catch (Exception e) { }
         }
      }
   }
}

start the following command line wt.exe sh -c 'echo abcd\n && read -p prompt' wt: sh:

all works fine:

now change && for ; -> start: wt sh -c 'echo abcd\n ; read -p prompt'

wt: sh: will fail

Now, I am able to take that somebody decided that semicolon ; as an argument is a good tab splitting character, however obviously this is problematic in context of standard *nix shells.

However, it is worse. WT ignores the "good citizens" rules about how command line is supposed to be parsed on windows (I really enjoyed reading this one again after long time). https://daviddeley.com/autohotkey/parameters/parameters.htm

WT interprets semicolon ; anywhere, in any argument, as command to spawn new tab, even if it is in a middle of "what should be parsed as a single argument" on windows!

repro steps: command line: wt.exe sh -c "'echo abcd\n ; read -p prompt'"

wt:

the argument gets torn apart by the semicolon and the argument fails to start

there should be just one tab, and the command line to start sh should like sh -c "'echo abcd\n ; read -p prompt'" (does not matter if sh can handle that well or not, see note bellow too)

also: command line: wt.exe sh -c 'echo abcd\n; read -p prompt' should work too, because the ; is not an isolated argument, but part of the argument abcd\n; So the command line passed used to start sh should be simply sh -c 'echo abcd\n; read -p prompt'.

note: sh is parsing the command line in a shell-way, that's why the argument with inner command line, passed after the -c argument, can be enclosed in single quotes '; However that is just a feature of this particular reproduction steps and does not affect the "core issue" in WT.

Expected Behavior

WT does not treat ; that are part of an command line argument (https://daviddeley.com/autohotkey/parameters/parameters.htm#WINCRULESCHANGE ,referencing MSDN docs) that contains other characters then the single ; (assuming spaces/tabs around arguments are trimmed by the command line argument parsing code) as create new tab commands.

Actual Behavior

WT treats all ; in all arguments as create new tab commands. As a consequence it is miss-interpreting arguments designed to reach "target CLI application". Such behaviour does not have any grounds in standard windows command line parsing procedures and leads to confusion and need for special workarounds, that would need to dynamically rewrite arguments being passed to WT.

DHowett commented 2 years ago

This seems very similar to #13255. We should consolidate them into one issue (prefer this one as it is more on-target.)

zadjii-msft commented 2 years ago

I agree.

Finding a way to split this up might be tricky.

What about something like wt -p "foo;bar"? That one seems like it would be looking up the foo;bar profile.
But then something like wt -p foo;nt - what should that do?
Or even just wt nt;nt;nt? I would think that should still get broken into subcommands, even though they'd all get parsed as one arg, nt;nt;nt

lhecker commented 2 years ago

In my personal opinion we should have as little surprising behavior as possible here. For instance if I'm scripting and I write wt -p $profileName in PowerShell I don't expect that I have to escape semicolons in the string first. I think that's a problem. 🤔

As such I'd say wt -p foo;nt looks for a profile called foo;nt and wt -p foo ; nt does the alternative behavior with new-tab spawning (assuming the shell doesn't parse semicolons). I don't think we can use quotes alone as a differentiator to decide whether semicolons are part of the string or not. After all, we don't exactly know whether wt -p $profileName always sends the variable quoted and will continue to do so in the future.

At this time I can think of two approaches:

A strict one: ; only acts as a seperator if it's a standalone argument in the argv array
A lax one: ; also acts as a seperator if it appears outside of quotes, while quotes are present (e.g. "foo";bar, foo;"bar", or "foo";"bar", but not foo;bar) - this one is still a bit "surprising"

michkot commented 2 years ago

Just so everyone is on the same boat - quotes are the thing that will make sure stuff will stay together in a single argv entry of application compiled with MSVC (compatible) compiler, on Windows - the compiler does the work of injecting parsing code that converts single-string windows commandline into an arg vector (aka. argv) for you.

edit; This was mainly pointed at lheckers last paragraph. I think zadjii-msft got it right; and brought an interesting question about wt -p nt;nt and wt nt;nt;nt - which is the core issue here actually; WT currently disrespects argument boundaries and treats ; anywhere in the "positional args" as a special thing;

The only fix IMHO is to not do that, meaning that wt nt;nt;nt would try to start a process with command line nt;nt;nt (would work if there us such executable file)

michkot commented 2 years ago

I just I though about this again. I realized that forcing "windows standard CLI parsing" might actually not rly improve life of people who regulary invoke complicated bash CLIs through WT - the need to wrap the whole inner command line into " quotes would clash with the " quotes used within the inner command line.

However I can imagine that people like me (which struggles with this ;-mess in automation scenarios) would be made happier with a "workaround" - an alternative WT command line parsing modes being added.

It would end-up having (maybe) 3 cli parsing modes "current one" "standard windows" parsing mode, wt -cw command-a first-param second;param third-param ; command-b "first ; param" second-param "raw string mode", ala raw string literals in C++/python" wt -cr MAGICRAW MAGICRAW command-a and its command ; line that does not end until I write MAGIC and RAW as a single word MAGICRAW ; command-b;command-c

... there could be alternations to this; The thing is, I am not that familiar with WT's CLI switches; Must all switches (like -p xxx come before all commands and command-separators? Or can they be interleaved? How do we know if it's a swtich to WT or parrt of the inner command line to start? Based on these thing, the "raw string mode" might be a great thing or a great overkill ; Maybe it would be enough to allow to specify custom command-separator character instead of the default";" 🤔

caioaamaral commented 3 months ago

[friendly ping]

I just found this problem while evoking commands in Ubuntu WSL:

wt -w0 new-tab --profile "Ubuntu" --title "Echo Hello" -- wsl -e bash -c "echo 'Hello World'; bash"

[error 2147942402 (0x80070002) when launching `" bash"']

but using && works fine:

wt -w0 new-tab --profile "Ubuntu" --title "Echo Hello" -- wsl -e bash -c "echo 'Hello World' && bash"

Similarly, same applies for envar acess with $:

wt -w0 new-tab --profile "Ubuntu" --title "Echo Hello" -- wsl -e bash  -i-c "echo '${HOME}' && exec bash"

# returns powershell $HOME variable (i.e "C:\Users\<username>")

wt -w0 new-tab --profile "Ubuntu" --title "Echo Hello" -- wsl -e bash  -i-c "printenv HOME && exec bash"

# returns bash's $HOME variable for the current user (i.e "/home/<username>")

microsoft / terminal