picture-in-a-picture by becoming a pseudoterminal

[dankamongmen]

Walking around the streets of plague, I had an inspiration. Rather than trying to munge a TUI subprocess down via LINES/COLUMNS or anything like that, which was never really going to fly, we can do this properly by providing pseudoterminal functionality. We'd launch the process with its stdout/stderr/stdin connected to a terminal, valid under istty(), but that terminal would be us. We'd export a locked-down terminal the size of the desired rendering area, with the capabilities we wish to support, and it would work perfectly.

BOOM.

Hell, at that point we basically can become screen/tmux.

Very exciting.

[dankamongmen]

This would be necessary for #381.

I have reached out to the NCURSES list. They're full of wisdom, when not too grumpy. Let's see if we hear back anything interesting. https://lists.gnu.org/archive/html/bug-ncurses/2020-11/msg00007.html

[ghost]

Are you looking for a terminal widget that behaves like "TERM=xterm" ? There is vte (current git head has sixel support), but I think that requires GTK. Here is one in C++ that relies on the Ultimate-C++ (U++) framework that also supports sixel: https://github.com/ismail-yilmaz/upp-components/tree/master/CtrlLib/Terminal . If you were interested in the latter I would be very unsurprised if İsmail helped in isolating it from U++ for other uses.

[dankamongmen]

Are you looking for a terminal widget that behaves like "TERM=xterm" ? There is vte (current git head has sixel support), but I think that requires GTK. Here is one in C++ that relies on the Ultimate-C++ (U++) framework that also supports sixel: https://github.com/ismail-yilmaz/upp-components/tree/master/CtrlLib/Terminal . If you were interested in the latter I would be very unsurprised if İsmail helped in isolating it from U++ for other uses.

Bryan Christ pointed me at this: https://github.com/TragicWarrior/libvterm. I think it would fit into our current idiom better. upp looks interesting!

[joseluis]

@dankamongmen I'm personally very interested in this feature and I'd like to discuss how much work would be necessary, and how hard would be to make this a reality? Is there any chance to add this to the 3.0 milestone?

As a tmux user/addict I'm sad about its lack of sixel support, which according to some info I read it's deeply ingrained in its design and wont probably ever change. And I don't want to wait years hoping for a viable alternative to come up.

So I would very much like to attempt creating a simple terminal multiplexer with notcurses, that allows me to manage multiple sessions, windows and panels with pixel support...

What are the missing pieces?

[dankamongmen]

@dankamongmen I'm personally very interested in this feature and I'd like to discuss how much work would be necessary, and how hard would be to make this a reality? Is there any chance to add this to the 3.0 milestone?

As a tmux user/addict I'm sad about its lack of sixel support, which according to some info I read it's deeply ingrained in its design and wont probably ever change. And I don't want to wait years hoping for a viable alternative to come up.

So I would very much like to attempt creating a simple terminal multiplexer with notcurses, that allows me to manage multiple sessions, windows and panels with pixel support...

What are the missing pieces?

well, there's the argument that multiplexing really belongs in the terminal itself, which i kinda subscribe to. that's why i closed up #1332 . this is more about processing a stream including escapes, and directly rendering it. we have ncfd_subprocess to soak up subprocess data, but if it includes escapes, we don't do anything with those.

as for how much effort....i don't know! there's the U++ tool @klamonte mentions above, and also libvterm. do i need some vterm terminfo database for the latter? i'm unsure; i've never really looked into this.

i don't see it happening in 3.0 unless

• someone pays me to do it, or
• someone writes it and the code looks proper

there's just so much crap scheduled for 3.0 already, and i want to have it out this summer.

[joseluis]

Thank you for the quick response... that helps framing the situation.

this is more about processing a stream including escapes, and directly rendering it.

If I wanted to to that part in Rust, (e.g. by forking vt100 to add support for the vt340 escape codes), am I right to assume that only missing part in notcurses would be to offer an API point in order to receive all the subprocess data so that I could then render it myself?

[dankamongmen]

https://notcurses.com/notcurses_fds.3.html ...although you don't want to render directly to a plane. so you just kinda want popen(3) heh, unless i'm missing something.

[joseluis]

Wow, I've realize I never used that before... In fact, I didn't even include it in the bindings. I'll have to remedy it.

Then I think I'll port the procroller. and fileroller.c pocs before attempting anything more ambitious....

[dankamongmen]

@tomek-szczesny , not sure if you've seen this issue, but it seems relevant

[tomek-szczesny]

Thanks for pointing it out for me. Well that explains why @joseluis lurks my repo :d

For me, equally exciting is a fact that @joseluis is tempted to write ntmux (not tmux), and I'm eager to write ncssh (not cssh). Two perfect examples of potential use and ultimate tests of new capabilities.

So, what do we take out from this, given all new information we generated in the meantime? I see the grumpy ncurses people told you to use libvterm and dropped the mic. The tty witchcraft will definitely be left to you. What are the other pieces?

(still waiting for your respone here, @dankamongmen )

[ghost]

Some additional references:

• (More for the audience here, since this is @joseluis 's project.) zellij is Rust, but is definitely a contender for a tmux successor. Its vt100 widget is in OK shape -- handles terminfo fine, but has some gaps in vttest. I'm guessing this might be the source of 'ntmux' ?
• If you want to start from a strictly C VT widget that lacks sixel support, lcxterm's isn't too awful IMHO. It delineates VT100, VT220, Linux, and Xterm, and passes vttest with VT220 (one small glitch in the VT52 subtest). It is unusual in that it follows the Paul Williams DEC state machine but at top-level is driven by a huge switch rather than a separate state table; I had intended at one point to condense to the state table everyone else uses, but it was actually more convenient to VT52 submode just to leave it this way. (I also have this in C++ and Java, which have the display factored out a little better from the VT. The C++ version was for RHTVision and is more ancient; I can provide a public domain licensed version if desired. The Java version is the most modern, and has sixel support). The comments may prove useful if nothing else.
• Once you have a VT widget, it is handy for both running a terminal inside (this issue), and also displaying pre-recorded "ANSI/ASCII art" screens - you get a "text file viewer" for free (just don't hook up keyboard input). That was really useful for offline mail readers back in the day.