search

oils-for-unix / oils

Oils is our upgrade path from bash to a better language and runtime. It's also for Python and JavaScript users who avoid shell!
http://www.oilshell.org/
Other
2.85k stars 158 forks source link

Shell as an engine for TUI or GUI #738

Open andychu opened 4 years ago

andychu commented 4 years ago

You may think this is already true: isn't a shell already a reusable component that runs under a terminal? You pass data on stdin, and get results on stdout and stderr?

No, because shell also has command completion and other UI features tightly integrated into it.

I guess this is an alternative to having something like ble.sh in-process. Shell could be a separate process that responds to completion requests.

So an interesting use case here is to make a GUI where the terminal line stays at the top? That is, the prompt and command line itself isn't part of the terminal. You edit it separately

related threads:

https://lobste.rs/s/brzfq6/does_anyone_use_mouse_with_terminal (e.g. Arcan subthread)

Playing With Debuggers (Zulip)

Videos about Terminals, Urwid, etc. (Zulip)

andychu commented 4 years ago

Shorter description: Oil could be a shell that runs in something other than a terminal.

The terminal interface is an optional layer on top.

@akinomyoga might be interested because this is an alternative architecture for something like ble.sh. Rather than having the line editor run in the same process, it could e a separate process controlling an Oil interpreter (two processes total).

I'd be interested in where that falls down. Of course it has to have a lot of hooks to expose its state. But you also have the benefit of not saving and restoring state.

I imagine we can come up with some sort of simple protocol over stdin/stdout. QSN #582 will definitely be a part of it.

problems: Signals need special handling. Would Ctrl-C, etc. translate to an IPC command, or would it have to be forwarded? The wrapper could "delegate" it.

andychu commented 4 years ago

Thinking about this a bit more, the signal issue is tough because you would want to simultaneously:

So basically this may involve changing shell to an async model, which is hard... But could be worth it. It's a big change.

The other option is to use a separate thread for processing IPC requests, but threads and fork don't mix... so this could be hard. Threads also introduce some runtime dependencies.

Third option: IPC commands could be accompanied by signals? e.g. the protocol is to send a signal, and the process knows to wake up and read a command from stdin? (or a socket, etc.)

andychu commented 4 years ago

760 for debugger support is related

andychu commented 4 years ago

Related to #663 too (provide APIs that allow people to write their own line editor)

andychu commented 3 years ago

822 is related (C++ code should be embeddable)

andychu commented 3 years ago

Previous discussions:

(Arcan) https://lobste.rs/s/brzfq6/does_anyone_use_mouse_with_terminal

(8 months ago) https://news.ycombinator.com/item?id=23521664

(October 2020) https://news.ycombinator.com/item?id=24873319

(January 2021) https://news.ycombinator.com/item?id=25925230

nyellin (Feb 2021): https://news.ycombinator.com/item?id=26123142

Some working code (in Rust):

https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui/topic/Lobste.2Ers.20followup

https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui/topic/async.20xv6

https://github.com/gmorenz/async-transpiled-xv6-shell

subhav commented 3 years ago

I'm absolutely interested in the idea of a GUI for a shell which runs outside of a terminal. If I were to prototype this, I might start with the popular Golang sh package, which I'm sure you're familiar with. It exposes a reasonable API for an interpreter with serviceable POSIX compatibility. However, proper support for Bash as a language, without its messy internals, would be extraordinarily useful.

Some off-the-cuff feature requirements from the shell might be:

andychu commented 3 years ago

Thanks for the feedback!

  1. Yes exactly, I see no reason the shell has to use the terminal. In general, child processes should be connected to a terminal (e.g. to get colors in your compiler output), but the shell doesn't.

I don't believe existing shells can be divorced of the terminal easily, but Oil is factored so it shouldn't be necessary.

I think you could also get far with the shfmt package, although Go has a runtime issue that would make me wary: https://lobste.rs/s/hj3np3/mvdan_sh_posix_shell_go#c_qszuer

Still that's not a reason not to try it and I would be very interested in the results of that experiment.

I just pinged Greg Morenz here since he has a promising experiment: https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui

  1. Yes for auto suggestions and fish-like syntax coloring, Oil can do this.

It's a similar "librarification" as the work above.

  1. Yes querying the PWD is a good point, and necessary.

  2. Yeah this is one of the most fiddly areas. I think it can be done and Oil is pretty principled about it. I just fixed a bug with signals where wait wasn't interruptible.

I think signals are the most compelling argument for the "IPC" approach, vs. the "API" approach. i.e. basically the UI could talk to the shell "server" over a socket or pipe.

Anyway I'd be interested to see an experiment with either shfmt or Oil... Oil has some weirdness where it's both Python and C++, we can talk about it :) shfmt is less complete as a runtime as far as I understand. I think Oil is significantly more compatible / featureful / etc. but I haven't looked at shfmt in awhile.

andychu commented 3 years ago

Oh here is the original discussion with Greg: https://lobste.rs/s/bl7sla/what_are_you_doing_this_weekend#c_f62nl3

I think this will happen "eventually", people just have to work on it :)

I think there is some value to running it in the browser, but I would like multiple clients, not just a browser one.

I think Jupyter uses ZeroMQ for all the fiddly IPC stuff? I wasn't sure I wanted to take that dependency but I'm open to ideas

Yeah I need to read this: https://jupyter-client.readthedocs.io/en/stable/messaging.html

I think there is a jupyter bash plugin but it's not very tightly integrated and I don't think it sees that much use (?) It probably doesn't have all the syntax analysis stuff. It would be good to do a comparison.

I guess the difference with Jupyter is that they're more like "one UI, multiple kernels" whereas this is sort of like "multiple UIs, the shell as a 'kernel' or library/server process"

subhav commented 3 years ago

Luckily, I don't care about POSIX compliance -- I just want a usable shell. But your goals are rightfully very different. :)

I use Fish on a daily basis, which is multithreaded and doesn't fork or really support subshells at all. I'm not a shell poweruser, but I haven't found that to be a problem so far. It means that stuff like echo (set foo 'bar') $foo totally works, but I don't know that that's a bad thing; it's at least more intuitive than what happens with subshells. Apparently subshell blocks are a feature in progress: https://github.com/fish-shell/fish-shell/issues/1439

That said, I think the shfmt package implements subshells by just creating a new Runner object for them in the same process -- no forking necessary, and it does have an exec implementation.

Oil as a Jupyter kernel would make a lot of sense!

There definitely do exist multiple Jupyter clients, though maybe they all share some internals. The IPython CLI, Jupyter Notebook, and JupyterLab are all different clients from the Jupyter team. Google Colab comes to mind as a third-party alternative, but I'm sure there are others. JupyterLab already has a "console" mode/document type, which is a little more ergonomically friendly than a notebook when all you want is an interactive prompt. But, I could see it making sense to build a lightweight client which is really geared towards shells.

The Jupyter Bash Kernel just uses pexpect and is pretty easy to break. :)

andychu commented 3 years ago

Just to summarize this conversation a bit more, we came up with the "entered command" abstraction, I proposed ui-eval, ui-wait, and ui-cancel builtins on the OSH side.

However those could maybe be called ecmd-eval, ecmd-wait, ecmd-cancel.

The core problem is the same one identified above (and partially addressed by the async xv6 experiment): the shell does a blocking wait() on processes, but it also would have to be able to process the ecmd-cancel message / builtin.

The GUI could send SIGUSR1 or something before every command to unblock the wait(); however it would also need to process IPC channel commands after being unblocked

https://oilshell.zulipchat.com/#narrow/stream/266977-shell-gui/topic/shell.20script.20as.20command.20server.20(Job.20Control.20without.20a.20Con.2E.2E.2E

subhav commented 3 years ago

How about interactive-eval, or something along those lines?

Whatever the API is, I think it's the facility that any kind of interactive client would be built on, including:

If you decide that this should be done using language features, I think it would make sense to dogfood it by ripping out main_loop.Interactive() and replacing it with an OSH script. :)

andychu commented 3 years ago

Yes that makes sense, I think it would be osh --server or osh --ui-server or something.

There should be another loop like Interactive(). It won't be replaced in the near term, but it would be an additional option

To solve the signal problem, I'm thinking that the dumbest simplest thing is just make the GUI send SIGINT when it sees Ctrl-C. It doesn't have to all be over and IPC channel

Commands go over an IPC channel, but signals are just signals. It's a different form of IPC. Thoughts?

(Another idea that might be overly general, it could be osh --coprocess, tying into the coprocess protocol I put on hold a couple years ago)

subhav commented 3 years ago

I'm not sure I understand what the --server flag changes. I understood interactive-eval to be a command that you'd put in a script with its own read/eval loop.

And yeah, having the shell forward a SIGINT to children that it's waiting on would work, I think.

andychu commented 3 years ago

Hm yes I guess there are still some open options on what to try:

  1. I was thinking of --server since there would be an IPC command to cancel, but if we just use SIGINT, then we don't really need that ?
  2. Writing the UI in Oil itself would be nice, but I think Oil doesn't have all the terminal bindings yet. So the UI process would be in Python or C or any other language to begin with. Writing in Oil, like ble.sh does in bash, is definitely a goal, but probably long term.

I still agree with the idea of every command having its own terminal, and the prompt being separate from the terminal, in the GUI.

Is that a concrete "next step" we can try?

I don't know exactly how to pass a terminal to the Oil process, which forks the processes that need the terminal. That is, I imagine the GUI creates a terminal and tells Oil to use it for the ECMD.

My initial thinking is to use a Unix domain socket because it supports descriptor passing, but I haven't tried that. Maybe it can just be a string like /dev/tty10 but I don't know exactly how that works!

This is all separate from the serial/parallel/state discussion so maybe we can figure it out first ...

andychu commented 3 years ago

I think at some point we talked about the vim/tmux issue, i.e. using the "alternate screen".

So in your view maybe the GUI doesn't literally have a terminal. It only has enough to be able to show "batch" commands and not vim/tmux/gdb?

I think that could be a worthwhile place to start as well. The issue is that isatty() would fail and then you wouldn't get color for compiler output, etc. right? Or shell output.

Say you start zsh or fish from the shell, or say you ssh to a machine with bash. You want to have some color I think

i.e. I guess this boils down to: is the GUI a terminal multiplexer or not? Making it one is obviously harder, but not making it one has some limitations.

(And it's different than a normal terminal multiplexer because it knows about prompt, and can know about state. The IPC command to dump state to the GUI is another reason I was thinking about --server, but that can also be a shell builtin that prints to stdout.)

This has a bearing on the API, but I think that if we use descriptor passing maybe Oil doesn't have to care? Oil can be passed an FD for a terminal, or an FD for a pipe, and it will work the same? So that API is general enough for both purposes? I'd have to play with it

random google result about descriptor passing, I think there are probably better ones: https://news.ycombinator.com/item?id=24964966

https://stackoverflow.com/questions/28003921/sending-file-descriptor-by-linux-socket

andychu commented 3 years ago

Thinking aloud a little more: the thinking behind --server is that it's "whatever is necessary to separate the prompt from the terminal". (which another person was pursuing as well)

The shell and the child processes share a terminal in all shells. This functionality isn't supported by -i or batch mode in any existing shell.

We could also call it --no-prompt but I don't think that's enough.

I also think there probably needs to be a distinction between "server/IPC commands" (interactive-eval, ecmd-eval, dump-state, etc.) and "shell builtins" (like eval).

The distinction is: regular eval sends its output to a terminal only. But I think dump-state has to send its output back to the GUI/TUI, not to a terminal.

In summary, reasons for --server, --coprocess, whatever:

  1. don't print the prompt to a terminal
  2. dump-state to communicate with GUI/TUI, rather than terminal for the user
    • also: results of completion queries go to the GUI, not to the terminal
  3. fd passing maybe, so each ecmd can have separate output in a GUI

Feedback welcome, this is all rough thinking, really needs a prototype!

subhav commented 3 years ago

Ahh okay, I was thinking that interactive-eval (or whatever) was supposed to be a shell builtin. My mind is stuck on the command server that I wrote for my bash prototype.

I guess you were thinking of it as some sort of IPC-specific method?

So in your view maybe the GUI doesn't literally have a terminal. It only has enough to be able to show "batch" commands and not vim/tmux/gdb?

In my view, it's up to the GUI/client what the standard io of a command should be. A GUI might want to allocate a terminal for each command, a simple prompt in a terminal would just always use that same terminal, a Jupyter kernel might want to just use pipes.

I was curious about passing fds around with domain sockets too, for the same reason haha. In my bash prototype, I have to create a named pipe in /tmp for stderr and have bash open it. I'm glad our thinking is kind of converging; it makes me feel a little less naïve about this stuff.

Although, we were talking about whether job control is really necessary. The idea of every command having its own terminal is only relevant if some commands might run in the background. If there explicitly won't be support for background jobs at all when running interactively, then maybe the command-stdin/out/err could just be passed to the shell as additional fds when the process is created and reused for every command.

andychu commented 3 years ago

OK I think I have an idea of how to do this, or at least get started. And a good name for it -- it should be --headless.

jupyter-screenshot

subhav commented 3 years ago

What would --headless do, exactly? Is that different from the normal non-interactive/"batch" mode? A shell running a script is already "headless".

Even if the shell is blocked with no job control, I still see a use case for terminal-per-command. You could do something like jupter where the prompt is separate from the command output. And you can put the prompt at the top, etc. The UI can block until the headless shell returns

Yeah, the standard IO for an evaled command should definitely be separate from the stdio for the shell (or whatever channel the shell is reading commands from). But if the command can't continue to run in the background, then the stdio for the command could be recycled for the next command.

Regardless, being able to change this per command is still the more generic interface.

andychu commented 3 years ago

It's just a name for what I described above: doesn't print the prompt, FD passing, etc.

I think it would have to be --headless /tmp/oil-socket. The FD passing requires a slightly different code path.

There's also the possibility of separating the shell's stderr from the command's stderr. For example, for syntax errors, if you type something like )oops, vs. ls producing an error.

I think it should be possible to prototype in Python, since the stdlib has all the syscalls: http://ptspts.blogspot.com/2013/08/how-to-send-unix-file-descriptors.html

I don't know exactly how to do the terminal thing; that's something we could talk about. I guess ssh -t does something like this.

This looks like a good start:

https://www.uninformativ.de/blog/postings/2018-02-24/0/POSTING-en.html

Also: http://neugierig.org/software/blog/2016/07/terminal-emulators.html

I don't want to write a terminal emulator, just demonstrate that it's possible to hook up a child process to a terminal passed over the --headless socket. That said I think there are a bunch of small terminal emulators out there

Hm actually there are several in Python here:

https://pypi.org/project/pyte/

https://github.com/selectel/pyte

Shows some syscalls: https://github.com/selectel/pyte/blob/master/examples/capture.py

https://docs.python.org/3/library/pty.html

andychu commented 3 years ago

Here is what I think the flow would look like:

  1. GUI process invokes osh --headless /tmp/oil-socket . It can have many of these running at the same time
  2. Then it enters a loop to write commands to /tmp/oil-socket.
    • one of the commands is probably to receive an FD and store it in a variable. So readfd in out err. The GUI sends the descriptor AND the command to tell the shell to read it.
    • Then if the user types ls /, the GUI can run something like ui-eval --stdin_fd $in --stdout_fd $out --stderr_fd $err 'ls /'. (Although care has to be taken not to clobber user variables; ble.sh has that issue)
  3. GUI listens for a "done" message on /tmp/oil-socket. When a given osh process returns "done", then it's unblocked and can run another command.
    • In particular, SIGINT should cancel the current ECMD and return "done".
  4. The GUI can send messages without user input. For example it can send a dump-state command to get cwd, ENV, $?, $PS1, ${FUNCNAME[@]}, etc. In this case, it doesn't use a terminal -- it can use a pipe that the GUI itself will parse from.

We can talk about concurrency more later, but I think this is a handful as is... i.e. might be months of work, and it could be marginally better / no worse than using tmux or xterm, etc. windows are blocked in those cases too.

(EDIT: I think this alternate --headless main loop should be fast to prototype; I meant an end-to-end demo might be months of work :) But hopefully we can figure out if the concept is worthwhile well before that)

Goal: separate prompt, history, completion from the terminal. These things could be nicer in the GUI, e.g. work more like the browser or like devtools. The browser's URL bar has all those things: prompt, history, and completion.

subhav commented 3 years ago

When the shell is in headless mode, does it only accept special IPC commands and only use the shell interpreter when you run ui-eval? Or, are ui-eval, readfd, and dump-state handled by the interpreter too?

Would calling dump-state modify the status in $?, for exmaple?

I wonder if it's worth taking inspiration from how REPLs are implemented for other non-shell languages, here. Or at least, I'm curious if you would know more about that sort of thing. I think the default interactive prompt in Python is built into the interpreter (here?) but there are also all sorts of other interactive python UIs, like bpython or ipython/jupyter. The command line interface for v8 apparently uses the same protocol to connect to the interpreter that devtools uses.

I'm having second thoughts about whether polling the state of the shell is really good enough or if the shell needs to push state changes to the client. For example, if you ran cd; sleep 10, the GUI wouldn't know about the directory change until the end of the command. Is that an issue?

Really random sidenote, but one interesting thing about descriptor passing that I found is that the file descriptor is allocated when you parse the out-of-band message from the read. So if you're using a stream socket and the message is fragmented, you need to make sure that you only read the OOB message for the first read and not the second. Otherwise, it allocates two fds for the same open file.

I don't know exactly how to do the terminal thing; that's something we could talk about. I guess ssh -t does something like this.

Allocating terminals is totally up to the GUI; the shell just needs to know what stdin/out/err to run a command with. But if you want to try it out, creating a pseudoterminal is just: fd = open("/dev/ptmx"), unlockpt(fd), and ptsname(fd) to get the corresponding pts.

I'm just using a fairly simple terminal emulator library for my prototype. I'm sure there are plenty of similar colorizing libraries for Python.

Whew, there are a lot of threads going on here. I hope that wasn't too all-over-the-place.

andychu commented 3 years ago

Sorry for the delay, I wanted to get a prototype going before getting to deep. I had an idea that I believe would be 300-500 lines of code, e.g. in a file core/headless.py, but I haven't yet tested that theory.

So let me try to answer the questions for now, with the caveat that it could all change based on a prototype. All of these are good questions!

  1. Yes a key question is if the IPC commands are builtins or not. It would seem "obvious" to make them separate, but I think combining them has advantages. I've learned the hard way that it's better to expose less surface area. And part of this is inspired by ble.sh, which is written in bash. If you haven't seen it already, I highly recommend this page written by @akinomyoga :

https://github.com/oilshell/oil/wiki/How-Interactive-Shells-Work

  1. The conflict with $? is a good point. Oil already has a specific mechanism for $?, because $PS1, $PS4, and $PROMPT_COMMAND have the same "clobbering problem.

https://github.com/oilshell/oil/blob/master/core/state.py#L874

But there could be other conflicts with state. That is why I mentioned #704 , the subinterpreters bug.

  1. I could imagine mechanisms other than polling, but it's a place to start. BTW I just learned that newer versions of tmux actually poll the process for the cwd! I'm imagining a GUI that's very tmux-like, and I think we can do better than what tmux does.

https://stackoverflow.com/questions/28376611/how-to-automatically-rename-tmux-windows-to-the-current-directory

set -g status-interval 1   # hacky asynchronous polling

If you have any running code for the descriptor passing, I'd like to see it! I haven't had time to play around with it yet.

(As for what I've been up to, I went on a nice tangent with git annex, which might play into the Oil project eventually)

But I hope that I can get something going with 300-500 lines of code... just to flush out some of the issues we've been talking about.

emdash commented 3 years ago

You might have a look at the pty module in python's standard library. It provides openptyand a couple abstractions built on top of this. What's nice about it is that it works across different flavors of unix -- namely Linux and OSX -- where there are some subtle differences.

If nothing else, the code would be worth reading the module source to see what calls it makes, and how it handles platform differences.

andychu commented 3 years ago

Yeah Oil's philosophy is to be completely portable. That's one reason there is a new headless shell feature:

http://www.oilshell.org/blog/2021/06/hotos-shell-panel.html#oils-headless-mode-should-be-useful-for-ui-research

The demo client uses the pty module:

https://github.com/oilshell/oil/blob/master/client/headless_demo.py

To write a bash-like TUI, I think you don't need anything unportable. Terminals on all Unixes are compatible in that respect.

But a richer GUI may want to take advantage of platform specific features. So it will be nice to punt those elsewhere!

If you want to give it a try, check out #shell-gui on Zulip and post a message!

dumblob commented 3 years ago

This might be a bit "too much" but there is quite recent and very successful effort SixtyFPS marrying a special declarative language (no DOM, no web s**t) with something-like-JS-lang to set the declarative part in motion.

https://github.com/sixtyfpsui/sixtyfps/tree/master/examples

Maybe Oil could adopt the declarative part and swap the JS-like part for Oil itself...