Documentation: terminal uses `;` for separating subcommands

ad-on-is commented 4 years ago

When executing the following commands, they don't work

wt ; split-pane -p "Windows PowerShell" ; split-pane -H wsl.exe or wt new-tab -p "Windows PowerShell" ; split-pane -p "cmd"

It opens only the first profile, but not the second one, and throws an error in the main window.

split-pane : The term 'split-pane' is not recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

DHowett-MSFT commented 4 years ago

If you're launching from powershell, it's interpreting ; as a command separator (much like bash). You'll need to escape it as `; or use the stop-parsing indicator %--.

zadjii-msft commented 4 years ago

I want to re-open this, since we should update the documentation here to mention the powershell quirks

maknapp commented 4 years ago

Why is ; being used to separate command line arguments when it is a command separator in both bash and powershell?

ad-on-is commented 4 years ago

@DHowett-MSFT yep, the issue lies within PowerShell, in cmd it works fine.

DHowett-MSFT commented 4 years ago

Valid docs issue for 1.0; yanking the triage tag.

@maknapp this is what tmux does. We're in an interesting place, being an application that spawns other applications (often from other applications) -- we need to choose a character that isn't going to be too likely to be used with the receiving application (the final shell spawned at the end) but doesn't provide a terrible experience for the spawning application. We think tmux did a pretty good job of that overall.

zadjii-msft commented 4 years ago

Additionally, we discussed this in the spec when this feature was being proposed and reviewed.

gwojan commented 4 years ago

Can someone explain why wt --% ; split-pane -p "Windows PowerShell" ; split-pane -H wsl.exe when executed from PowerShell blocks the launching PowerShell session until the newly created Windows Terminal window is closed?

I don't want to hijack this issue but was wondering if it's just another documentation issue. Should I open a separate issue?

DHowett-MSFT commented 4 years ago

I sure can! That's because of https://github.com/PowerShell/PowerShell/issues/9970.

alexandair commented 4 years ago

This is not a documentation issue. Windows Terminal is a Windows app. The most popular shell on Windows is PowerShell. And, we need to use escaping character to make wt.exe's command-line parameters work in PowerShell?

Jaykul commented 4 years ago

Let's be honest: I never saw the spec until now, but this is hard to use (from PowerShell), and nearly impossible to script (since there's no console output) and you obviously knew that going in.

I don't understand the whole "this is what xterm did" argument, but it doesn't hold water. Did you write a man page? This is not unix, and it's not bash, and xterm isn't a "window mode" app that pops up dialogs for help, either. The semicolon is a command separator in a lot of shells, and particularly in the primary modern shell on the only operating system where this application runs.... I think you picked the wrong separator. 'nuff said on that.

I also think the output going to a dialog is the wrong choice. Features like list-profiles and version are simply not useful if we have to read the output with our human brains from a dialog. I can't write a script to check that, so I might as well just fire up the app and hit the menu for a list of sessions, or hit help -> about for the version.

Frankly, I have not even tried to use wt until today, because the --session functionality still hasn't been implemented, so it always opens a new terminal, which is simply not useful at all to me. I don't know if there's anyone running lots of windows instead of tabs, but that's not me. If Windows Terminal wasn't tabbed, I'd still be using ConEmu.

I'm sorry I (or "we") didn't get involved in the design spec stage ...

For what it's worth to the docs: the only syntax that works at this point without complications in PowerShell to use start-process and quote all the parameter blocks. Even then, you must be careful to use double quotes around profile names (not single quotes, because it can't handle it), and you can't use the normal double-double method of escaping so you have to write one of these two syntaxes:

Single quoted parameters (if you aren't calculating anything):

start wt '; split-pane -p "Windows PowerShell" ; split-pane -H wsl.exe'

Escaped quotes (if you need variables):

$ThirdPane = "wsl.exe"
start wt "; split-pane -p `"Windows PowerShell`" ; split-pane -H $ThirdPane"

Because if you just escape the ; your current instance is stuck

wt new-tab "cmd" `; split-pane -p "pwsh" `; split-pane -H wsl.exe

maknapp commented 4 years ago

I would suggest using ! instead of ;. It appears to work fine as arguments in cmd and powershell and is fairly uncommon.

DHowett-MSFT commented 4 years ago

@Jaykul so, there's a couple issues here

`;` as an argument separator

The team went back and forth on this for quite some time. There's always going to be a character that's not going to work for somebody, and we had to choose one.

You're not going to like me drawing a parallel here with unix, but I'll get to why I think it's germane to the discussion. We considered tmux to be the closest analogue to Terminal: it has a remote control protocol, it is a terminal multiplexer that supports split panes and "tabs" (though they're implemented as full-screen windows), and it has a terrifically scriptable interface using a single command language that's accepted either in arguments or in a config file.

That's exactly what we're going for here.

The reason I'm bringing it up again is that they chose ; despite the shell users use to typically host it: bash. It could have been any shell, though--tmux could have grown up on Windows where PowerShell was the shell du jour--but still they chose something that was familiar/native/natural to their users. "I want bash to do multiple things" is the same as "I want tmux to do multiple things", which is the same as "I want powershell to do multiple things" and "I want Terminal to do multiple things." It's a minor cognitive overhead compared to having yet another command separator, isn't it?

I still can't remember how to run multiple commands on the same line in cmd. I've been a Windows platform resident for most of my computing career, and it just doesn't stick with me. Should we have introduced something else, like !? *? ^? Somebody, somewhere, would be using those in an environment where they can't be passed to Terminal without escaping them. Since we had to choose something from a limited set of characters, we opted to choose the one most familiar.

Displaying output in a dialog

This is, perhaps, the worst travesty Windows has foisted on commandline applications. I'll start from CreateProcess . . .

Broadly, there's two major application subsystems in use on Windows today, CONSOLE and WINDOWS. I'm going to stop capitalizing them and putting them in preformatted code blocks, because it's going to get tiresome fast.

Console Applications

When spawned, kernel32!DllMain (this is an approximation) will check whether the application has inherited a console reference handle.
If it has:
- Standard in, out and error will be connected to that handle.
If it has not:
- A console driver connection will be made and conhost.exe will be spawned to service the server side of that connection.
The application will proceed on its merry way.

Windows Applications

None of the above considerations matter in the least.

All process spawning is asynchronous in Windows, as it is on various POSIXlikes. That presents a challenge for a console subsystem application, which would immediately connect to a a console, even if its parent was already connected. So... there's some weird magic that cmd and powershell do to handle these different types of applications.

Both cmd and powershell crack the PE header of the executable on disk and ask what subsystem it is. If it's a console application, they wait on the process handle. If it's not, they don't. You get the right behavior for most applications: notepad returns you to your prompt and python doesn't.

Consider, though, the application that wants to both be graphical and present output to a console. How does it work?

Well, here's how:

Python ships two binaries, python and pythonw. pythonw is one subsystem, and python is the other. Forevermore on Windows, developers must know that python whatever_ui.py spawns a strange and confusing black window in addition to the actual application window, and pythonw provides them literally no feedback when spawned from cmd or powershell. It's atrocious, but that's what we have today.

WT could ship another app execution alias, wtw, that follows the python model. That's equally atrocious, isn't it? wt is easy, but if we made it a console subsystem application simply so that cmd.exe would wait for it to finish printing whatever it wanted to print, it would spawn a black box every time somebody launched it. Well, two, because it is itself a black box. :smile:

I explicitly asked @zadjii-msft to go with the dialog box solution (and eventually transition to the msiexec /? solution, where it at least pops up a dialog box with scrollable copyable text) because we just can't afford, in the very literal sense of both monetary and political capital, to push another subsystem with the compiler and linker and OS and the PE/COFF format (which is now also part of the UEFI specification) or standardize yet another workaround that'll only be available in two or more years.

None of it's pretty, but we're working at the intersection of expected norms and literal ground truths here. I'm sorry that it's not better.

DHowett-MSFT commented 4 years ago

(We can probably do some math on our standard output handle, if we have one, to see whether it would be amenable to receiving output. This would at least allow wt to be used in a scripting context, even if it couldn't be used to literally print wt --help to the console. Trust me: we've spent almost eight hours in discussion alone about exactly how tied our hands are.)

jantari commented 4 years ago

Couldn't you embed the "real" windows terminal program as a resource inside a console mode exe that only calls the embedded resource when it's not called with --help or similar?

DHowett-MSFT commented 4 years ago

@jantari that's got all the same problems as just shipping wt and wtw (where wt calls wtw or something), except with the added complexity of unpacking an EXE and executing it (which will make us look a lot like malware.)

If the wrapper is a console mode exe, kernel32!DllMain will allocate a console. If it is a windows mode exe, cmd/powershell/bash will immediately return to a prompt before it has a chance to print any output.

Jaykul commented 4 years ago

@DHowett-MSFT ok, you know what ... I can live with ; or rather `; -- I mean, it makes sense if I remind myself I'm passing a script into it.

But it's just not the same. At all. From WSL pwsh, this works as expected. It makes a new session and connects it in the terminal I was already in:

tmux new-session bash `; split-window pwsh -noprofile `; split-window -h cmd.exe

But the equivalent wt command ... does not:

wt new-tab wsl `; split-pane -H pwsh -noprofile `; split-pane cmd

Opens a new window instead of using the one I'm in. And just a tab as I expected.
Locks up the tab I was in
Works less than half the time. I usually only get the first two panes. Sometimes only one.

Anyway. I've updated the post I wrote above with the `; example, but I don't think it's every useful. There's close to zero usefulness in running a wt command from within PowerShell without start (which will force using all that quoting).

On top of that, I can't use this syntax in a terminal "profile" for the same reason ...

riverar commented 4 years ago

; as an argument separator

The team went back and forth on this for quite some time. There's always going to be a character that's not going to work for somebody, and we had to choose one.

Why doesn't wsl just accept regular command line parameters like normal tui apps do? Wouldn't that be the most expected and compatible method of handling this?

e.g. wsl --split a --split b --split c?

Am I missing something?

Jaykul commented 4 years ago

Yes, @riverar, you are missing something. It is meant to take a script. Right now, it's parser and implementation are buggy, but look at a simple example. Run these two and notice that the order matters?

I've used PowerShell syntax:

wt new-tab -p PowerShell` Core`; split-pane -H -p Windows` PowerShell `; split-pane -p cmd

wt new-tab -p PowerShell` Core`; split-pane -p cmd`; split-pane -H -p Windows` PowerShell

This is like netsh or choco or any of dozens of sub-command commands, but the difference is that we need to be able to pass a script consisting of multiple sub-commands -- even the same sub-command multiple times (possibly with different arguments). The order matters, and all of the parameters that use -dashed syntax are parameters to the sub-commands.

Now, the parser they're using is not good enough. It can't handle writing these the way tmux users do, with new-line wrapping. It also can't handle when PowerShell starts adding quotes -- so in PowerShell, I'm forced to backtick escape spaces in profile names, rather than quoting them. But this doesn't work anyway, because it reads the newline as part of the command name:

wt `
new-tab -p PowerShell` Core`; `
split-pane -H -p Windows` PowerShell `;`
split-pane -p cmd

riverar commented 4 years ago

@Jaykul I don't see anything in your script that can't be expressed via parameters? What am I missing?

Jaykul commented 4 years ago

I'm not going to say it's not possible to do it as parameters, but I'm not sure what difference it makes.

The core requirement is to be able to split any pane vertically or horizontally, recursing indefinitely.

The current syntax seems designed to emulate tmux's syntax, which is based on interactive use. That is, tmux has a prompt you can enter with Ctrl+B, : and then they have commands you can run in that special prompt. Their command-line syntax is basically those commands, separated by the shell command separator.

wt has no such interactive prompt (so far), and the "commands" here are being created for the purpose of this syntax. I don't know if there's a plan to also expose a wt prompt so we can use those commands interactively, or not.

If not, I think choosing the command syntax is just an affectation 😉

To my mind, there's not much difference between the two, but I think if we leave off the command/separator notion, we would need to enclose parameters to the shell / profile in quotes.

DHowett-MSFT commented 4 years ago

I marked #4601 as the discussion parent for this issue, so I'm going to continue treating this one as "we should better-document the use of ;."

DHowett commented 3 years ago

@Jaykul So, I wanted to circle back with the benefit of some time having passed.

Opens a new window instead of using the one I'm in. And just a tab as I expected.

Locks up the tab I was in

Works less than half the time. I usually only get the first two panes. Sometimes only one.

In the past year we added windowingBehavior and the -w option to fix (1), worked with PowerShell to completely fix and also implemented a local workaround for (2) and think that we made pretty good progress on (3).

Does that track? :smile:

DHowett commented 3 years ago

Ah, and:

[editor note: 4] I don't know if there's a plan to also expose a wt prompt so we can use those commands interactively, or not.

Implemented the command palette with a commandline mode to fix (4) 😁

VanVan commented 1 year ago

Or maybe we could have made Windows terminal compatible with a classic list of arguments and the two most common arguments separators that all OS and program use, the space or the comma. No need to separate the list of arguments ( --help --maximized --fullscreen --focus --window ) from the others.

To be able to indicate a command like:

Replacing

wt new-tab "cmd" `; split-pane -p "pwsh" `; split-pane -H wsl.exe

By

wt -new-tab "cmd" -split-pane "-p pwsh" -split-pane "-H wsl.exe"

A variant with an equal symbol and a space in the sub-argument names:

wt -new-tab="cmd" -split-pane="-p 'Windows PowerShell'" -split-pane="-H wsl.exe"

A complex one works well:

wt -new-tab "cmd" -split-pane "-p 'Windows PowerShell' --title 'My Name'" -split-pane "-H wsl.exe --tabColor '#FF0000'" -new-tab "-p pwsh" -focus-tab 1

This makes it usable from PowerShell, Cmd, Bash and all others without escaping the characters, and allows not to confuse a Shell command list with parameters of a program, as well as to allow the launching of Windows Terminal with multiple command and classic ";" command separator

wt -c "ipconfig; pause"

Jaykul commented 1 year ago

Ah, and:

[editor note: 4] I don't know if there's a plan to also expose a wt prompt so we can use those commands interactively, or not.

Implemented the command palette with a commandline mode to fix (4) 😁

Let me preface by saying: I'm mostly happy with how things are going. But no, you did not. 😣 The same commands don't work in the "palette" at all.

We can't write new-tab or split-tab, we have to learn new commands.
The palette doesn't support my named profiles.
The things in the palette don't take parameters. We can't specify BOTH the shell and the split direction.
We don't learn the CLI syntax from the palette.
I wish the palette had the parameter feature from the CLI
I wish the CLI could copy syntax from the palette

I'm not sure how people are meant find out about the -H feature, since it's not in the help message from wt -?

DHowett commented 1 year ago

no, you did not

We sort of definitely did, it's just not the most discoverable feature we've ever authored[1] :smile:

It even tells you what it's about to do! With named profiles and all! It very much answers at least half of your bullet points.

... copy syntax from the palette

I like this.

1: it all starts with you having to delete the > pre-typed into the command palette; this "raw command mode" is conserved across a couple editors like VSCode and, I believe, Sublime

Jaykul commented 1 year ago

We sort of definitely did, it's just not the most discoverable feature we've ever authored[1] 😄 1: it all starts with you having to delete the > pre-typed into the command palette; this "raw command mode" is conserved across a couple editors like VSCode and, I believe, Sublime

OMG, you did! That's awesome! 😯🤨😑😣😞

Too bad you buried it without so much as a grave marker 🪦🪦🪦

I think this team has some misunderstandings about how command palettes work everywhere else, which I can't understand. First of all, these are not a new idea in VS Code (or Sublime). Please have a look at some of the "launchers" that introduced the concept so many years ago, like Alfred, Albert, Launchy, Krunner, Quicksilver, (and ... Flow and PowerToys' Run).

The `>` in all of these tools is a LITERAL character in the command you enter

In other tools, I can start typing and then change my mind about the mode, hit "home" and "del"ete the > ... or even type a different mode character ... without loosing everything else I had already typed 🥺

In VS Code, if you delete the > then you're searching files in the workspace by name. It's not a "raw command" mode at all (except in the sense that the first character switches modes, so you can go into any mode). The > character makes it command-mode. The @ character is for navigation, etc. In launchers that first word invokes a plugin. E.g. in Flow I think > turns on the shell command extension by default, and in PowerToys Run it actually turns on ... a Windows Terminal extension.

If I was designing a command palette for this, there would be a lot more help in that bare mode (intellisense suggestions for commands, basically). The ">" would stay --literally-- in the command bar for the other mode. And the bar would never clear all the text on backspace ... unless I had selected all the text.

Anyway, I'm really happy to discover this is here ...

zadjii-msft commented 1 year ago

You know, there was some reason why we couldn't make it a literal character in the input, and instead faked it, and I can't remember the exact reason why now.

Also, we probably should support Home, Bksp to switch into commandline mode, and typing a > at the start of commandline mode to persist the rest of the input. I'm pretty sure there's a bug floating around here for that.

It will try to parse what you type:

and there's #7570 for "intellisense" in that box.

microsoft / terminal