iTerm2-like coprocess support

OceanS2000 commented 3 years ago

Is your feature request related to a problem? Please describe.

Currently there is no way to use lrzsz in kitty without GNU screen. lrzsz is a really old tool to transmit files over tty based on ZMODEM protocol.

While kitty has support for remote file transfer via ssh, sometimes I still finds lrzsz handy because it just needs a raw serial line and therefore works with nested ssh sessions or exotic systems reached with physical serial ports.

Describe the solution you'd like

For ZMODEM to work, the terminal needs to provide a mechanism to access the pty backing current screen. Currently, SecureCRT and GNU screen have specific support to it. But I believe iTerm's coprocess approach best fits into kitty's design.

A coprocess, as used in iTerm2, is a special process running in parallel with the terminal emulator that gets terminal input as its stdin and puts its stdout into the terminal. Supporting coprocesses not only enables using programs works over serial line (like lrzsz), but it also gives other possibilities. For example, when there are enormous log coming to terminal, I can have a process to recognize some regex and notify me, instead of having to redirect to a file and search in it later.

Describe alternatives you've considered

Kitten

Currently kittens are not capable to interact with underlying pty object. In main function it is running in an overlay window and its output does not write into the pty, and later in handle_result it can obtain the file descriptor of the pty by something like

def handle_result(args, data, target_window_id, boss):
    w = boss.window_id_map.get(target_window_id)
    if w is not None:
        fd = w.child.child_fd

But trying reading from this fd is a race condition with kitty itself.

Just use GNU screen and forget about it

Screen is somewhat still useful as it can dial into serial ports. But in general it quite clumsy to use and I rather use socat. Moreover, coprocess support is more than this rather exotic use case. From my point of view it gives kitty more extensibility and is benefit to other users as well.

Additional context

I would like to draft a PR for this but it is a rather major change (likely to introduce new API to kittens and kitty command line) so I think I should open an issue to discuss first.

If accepted, it will take some time as I am not very familiar with core kitty code.

kovidgoyal commented 3 years ago

I dont quite follow your use case. I can see two things you want to do:

1) Transfer files over the pty. This is best achieved by using an escape code tp transmit binary data. For example the OSC 52 escape code which kitty supports for transferring data to the system clipboard

2) You want to monitor the output from a log in the termial and get alerted on some expression. This is already achievable in kitty using marks: https://sw.kovidgoyal.net/kitty/marks/ which highlight matches for a regex. You can supply your own python function for it which instead of highlight or in addition to it does whatever you want, like launching an alert program

If there is some other use case you have in mind, please elucidate.

OceanS2000 commented 3 years ago

Transfer files over the pty. This is best achieved by using an escape code tp transmit binary data. For example the OSC 52 escape code which kitty supports for transferring data to the system clipboard

This is exactly what lrzsz is doing --- encode and decode binary files into a sequence of escape code by ZMODEM protocol. The point is that lrzsz works in both directions. I can invoke rz (the receiving end) on the remote system and let screen bring up sz (the sending end) on my local system and upload my file. With OSC 52 only downloading is possible as only the local terminal emulator understand it.

Another point is that lrzsz divides file into packages to transmit and provides error detection and re-transmitting of corrupted packages, which is a thing on physical serial line. But because of so it have to work interactively, and this is why I want raw access to the pty instead of save output of sz and paste into terminal later.

This is already achievable in kitty using marks: https://sw.kovidgoyal.net/kitty/marks/ which highlight matches for a regex. You can supply your own python function for it which instead of highlight or in addition to it does whatever you want, like launching an alert program

Thanks for the suggestion! I definitely will look into how to use this feature in my workflow.

kovidgoyal commented 3 years ago

Hmm, well I think I am going to decline to add this to kitty. Of the two use cases, monitoring the terminal output for expressions can already be done by marks. Transferring files is done by SSH, completely transparently in the common case using either the remote file kitten or a little ssh configuration and if one wants to use ZMODEM, one can, via screen. Given that adding the code to kitty seems not worth it.

OceanS2000 commented 3 years ago

That's fine. Still I find screen very clumsy and I may implement it in my private fork. Thanks for your attention all the same!

Luflosi commented 3 years ago

I once used minicom to transfer a file over serial to an embedded device using the XMODEM protocol, I think. It can send and receive files with the ZMODEM protocol as well. Does this work for your use-case?

tmccombs commented 3 years ago

Transferring files is done by SSH, completely transparently in the common case using either the remote file kitten or a little ssh configuration

The remote_files kitten has some limitations:

It doesn't work over nested ssh
It requires establishing a separate ssh connection
You either need to provide the hostname again when running the ktiten, or first generate a hyperlink like with ls --hyperlink=auto
The hyperlinks generated by ls --hyperlink don't work if the the hostname/ip you use to access the host doesn't match the host's local hostname (often the case for me).

An inline file transfer method would avoid all of those problems, but may introduce other ones. I'm not sure the coprocess method is the best way to do that, although it might make it easier to implement a tmux control mode client in a kitten. Especially if you could also create a new window/tab with a PTY and control it directly from the kitten without needing to run a separate process and communicate with it to control that that terminal.

kovidgoyal commented 3 years ago

tmux control mode is never going to happen, my views on tmux are well known by now I hope. As for inline file transfer, it's already in the process of being implemented in master, as I said I would in #3945

tmccombs commented 3 years ago

tmux control mode is never going to happen

sorry, if I wasn't clear but I meant as a third party kitten, not part of kitty itself. I'm aware at least some of your views on tmux, and if you don't want to build support for tmux control mode into kitty that's fine with me. I think it makes more sense for such an integration to be an extension rather than part of the core app anyway. I have an idea on how to implement most of it, the one piece I'm not sure how to do is take control of an existing terminal to communicate with tmux -CC that may be running over an ssh connection, or other remote connection.

kovidgoyal commented 3 years ago

On Wed, Sep 22, 2021 at 01:00:49AM -0700, Thayne McCombs wrote:

tmux control mode is never going to happen

sorry, if I wasn't clear but I meant as a third party kitten, not part of kitty itself. I'm aware at least some of your views on tmux, and if you don't want to build support for tmux control mode into kitty that's fine with me. I think it makes more sense for such an integration to be an extension rather than part of the core app anyway. I have an idea on how to implement most of it, the one piece I'm not sure how to do is take control of an existing terminal to communicate with tmux -CC that may be running over an ssh connection, or other remote connection.

I doubt kittens will be sufficient for this. Your best bet is to implement a wrapper program that sits between tmux and kitty and reads the tmux control mode codes and translates them into kitty remote control escape codes. It would also need to filter out any kitty remote control escape codes from tmux so as to not allow programs running inside tmux from using remote control.

kovidgoyal / kitty