Float window

[fedoranvar]

Good day to you!

Is it possible to toggle popup window like in tmux?

to be specific, to popup window with shell command on start

Thank you!

[imsnif]

That would be pretty cool. I think @a-kenji is working on this?

[a-kenji]

Yes, that would be awesome. I am definitely looking forward to something like this. I am working at something that could enable a floating terminal in the future.

[ghost]

If you choose to do floating window, then also consider implementing translucency. You could do some pretty fun things then, such as a guake-style dropdown terminal over the others.

It might be very straightforward, depending on your rendering model. Some notes here: https://gitlab.com/klamonte/jexer/-/issues/88

[imsnif]

I'm actually working on floating windows as we speak :)

Thanks for the interesting read! My main concern with this is that it might be a little difficult with eight-bit colors (which Zellij also supports). Wdyt?

[ghost]

You can use OSC 4 to query the terminal and get all of its assigned colors, and then at blending time convert the palette index to RGB. That's what these kinds of checks are doing in my code:

                    Cell cell = logical[col][row];
                    int thisBg = cell.getBackColorRGB();
                    if (thisBg < 0) {
                        if (backend != null) {
                            thisBg = backend.attrToBackgroundColor(cell).getRGB();
                        } else {
                            thisBg = SwingTerminal.attrToBackgroundColor(cell).getRGB();
                        }
                    }

...basically look for an RGB value, if that's not there (-1) then use a color based on palette index. If there is a connected user-facing head then backend is assigned and I have colors from a prior OSC 4 query; if there isn't a head then I use the color my own GUI head would have provided (which is usually DOS colors, but also user-assignable from my end).

But as @dankamongmen points out, those colors can be changed by the user after zellij is running, so maybe some periodic re-checks if issue reports come in.

[imsnif]

Thanks for the explanation. So a question: I take it the transparency is only for the background of cells and not for their content, right? A cell with an a character covering a cell with a b character will only have the a character in it. There isn't anything fundamental I'm missing here?

Otherwise, this sounds great but I think we'll need to do some ground work ahead of this. Zellij currently doesn't differentiate between clients who support truecolor and clients who don't. This means there are some potential edge-cases in which this is a problem (namely two clients with different capabilities having the same content rendered to them, but that's unrelated to this issue).

I think in order to support this feature we'll have to first log these capabilities and make the differentiation in order to handle the transparency properly. That or at least have this feature disabled by default behind a configuration flag. Wdyt?

[ghost]

You have it right: transparency is for the cell background (and foreground, if the underlying layer's glyph shows through), but not the glyph itself. If 'a' covers 'b', then only 'a' gets seen.

That's a good point about different client capabilities. The Linux console for example will parse 24-bit color sequences, but will map them to the nearest 8- or 16-colors on the VGA display; so it looks like you have truecolor but you don't have truecolor, and translucency will not work (and look awful). Older versions of xterm would map truecolor to the 288-color palette, which would be alright but again not look as good as the user might want.

I like your idea of tracking per-client capabilities, that's a general-purpose solution for many features. (It would greatly help the image support request : I recently wrote up some notes on image support to multiple clients with different capabilities and cell aspect ratios.)

Keeping it disabled by default sounds quite reasonable too. I don't know your composition model -- if it's top-down like notcurses, bottom-up like mine, or another way entirely -- but there are benefits and limits to both approaches. Top-down would permit many many more layers with high throughput, but the "stacking problem" as @dankamongmen points out can be difficult. Bottom-up is easy to reason about, but after about 50 layers performance starts to hurt -- even for text-only displays. If you get fully stacking floating windows with a nice UI, you would pretty much overnight become the most popular stacking-window terminal multiplexer. So it would be fun learning what the userbase really wants. :-)

[ghost]

(And also as a general rule: I'm sorry if me poking around is annoying to anyone. I like these projects and learn a lot from y'all, and try to share some tricks I have found along the way. But this is your project, and your decisions are totally great -- as seen by all the users! :-) )

[imsnif]

That's a good point about different client capabilities. The Linux console for example will parse 24-bit color sequences, but will map them to the nearest 8- or 16-colors on the VGA display; so it looks like you have truecolor but you don't have truecolor, and translucency will not work (and look awful). Older versions of xterm would map truecolor to the 288-color palette, which would be alright but again not look as good as the user might want.

Ah, good info. I think in general it would make sense to keep track of these capabilities (as well as support for the various image displaying protocols as you mentioned) and render our state appropriately to each client so that it'll get the best result. I have a long term dream of being able to support all the image displaying protocols and switch between them according to client capabilities - but maybe we can start by actually supporting one of them and go from there. :)

Keeping it disabled by default sounds quite reasonable too. I don't know your composition model -- if it's top-down like notcurses, bottom-up like mine, or another way entirely -- but there are benefits and limits to both approaches. Top-down would permit many many more layers with high throughput, but the "stacking problem" as @dankamongmen points out can be difficult. Bottom-up is easy to reason about, but after about 50 layers performance starts to hurt -- even for text-only displays. If you get fully stacking floating windows with a nice UI, you would pretty much overnight become the most popular stacking-window terminal multiplexer. So it would be fun learning what the userbase really wants. :-)

Amen to that :) By composition model do you mean regarding pane/window stacking and deciding what we render to the client? Since I'm just developing floating windows right now I'm just about to reach the point where I need to make this decision. My current gut feeling says it would be best to use a top-down approach for now, but I have not yet had the chance to devote much thought to the potential downsides. More reading material about said stacking problem would be greatly appreciated!

(And also as a general rule: I'm sorry if me poking around is annoying to anyone. I like these projects and learn a lot from y'all, and try to share some tricks I have found along the way. But this is your project, and your decisions are totally great -- as seen by all the users! :-) )

Please do not be sorry! Zellij is a community project belonging to its contributors and not to any one person in particular. Those who adhere to our Code of Conduct are more than welcome to participate, offer suggestions and share their experience. On a personal level I can say that I'm very happy every time I see your name come up in my notifications. I've learned a lot from your experience and insight and would very much like to learn more. Please keep participating.

And also, you are very much invited to drop by our Discord/Matrix chats if you'd like to talk more about the project or terminals in general.

[dankamongmen]

Amen to that :) By composition model do you mean regarding pane/window stacking and deciding what we render to the client? Since I'm just developing floating windows right now I'm just about to reach the point where I need to make this decision. My current gut feeling says it would be best to use a top-down approach for now, but I have not yet had the chance to devote much thought to the potential downsides. More reading material about said stacking problem would be greatly appreciated!

while i of course defer to @klamonte, i'm pretty certain that by "composition model" she effectively means "those things done by a compositing window manager, applied to the terminal". this IMHO implies the existence of a window type retaining its own data (data which might never actually be displayed), some z-order among a set of (possibly overlapping) windows to be composited into a visible frame, the semantics between corresponding cells of different windows (a terminal analogue of "alpha blending"), and the means by which that composition is effected (i.e. how to turn this data structure, at any given time, into an encoded update of the terminal, whether via full or partial redraw),

by top-down vs bottom-up she refers to the most fundamental operating principle of this fourth component--does it start at the lowest window(s) and work up, or contrariwise? @klamonte works bottom-up, using (loosely) what's known as the Painter's Algorithm. i work top-down, using (loosely) what's known as the Reverse Painter's Algorithm, though i personally think of it as the Algorithm of a Sensible Painter who has Other Things to Do That Day.

[ghost]

@dankamongmen has it exactly right. I chose painter's way way long ago because it was easy to think about, and I wanted to see where the limit was before performance started to hurt. It's given me opportunities to locate bottlenecks far upstream of generating the screen update sequences. (This is all over my codebases: get it working first, and then let time sort out the crappier parts to optimize.)

My windows have no individual screen buffers of their own: I set clipping/offset on the shared logical screen, tell a window to draw, and as it draws it is setting clipping/offsets on the screen for each of its widgets; then reset clipping/offset for the next window on top, tell it draw, and so on. Easy-peasy. But not-great for performance when there are lots of them stacked, and really-not-great when there are image cells underneath other things.

If one went the other way, and it was text-only, and there was no translucency, then it would be only a little more complicated, use a bit more memory (every window or plane needs its own screen buffer) but a lot faster. For each cell on the logical screen, walk down the layers to find the first that covers that spot, use that, and move on. Very similar to a Z-buffer algorithm. It gets more complex when you have transparency/translucency, as you need to go down several layers and include their contribution to the final fg/bg colors at that spot on the screen. That is (as I understand it) the heart of the stacking problem: just how far down do you have to go until you know that everything underneath that cell (or pixel!) is irrelevant?

I do really love that we have two such different approaches to play with. I've got a desire someday to try it the other way and see what else comes up, and whether it impacts other layers. Maybe even dynamically switch between them based on scene complexity.

[imsnif]

Hey, thanks both of you for all the great info. This makes a lot of sense to me now. I'm still at the point on my branch where I'm just rendering everything (in order so it makes sense on screen), but when I get to the end I'll decide between the two painter algorithm approaches.

I'm also curious if one of you has special approaches regarding output buffers? Specifically, say I have a complex stack of panes on screen and a few characters change in one of them. Some of those characters are visible, some are covered by another pane. Do you have a clever way of rendering just those changed visible characters to the terminal? Or just the usual "loop through cells, query all panes with greater z-index to see if anything is covering them" approach?

Also, while we're at this: could you recommend a good algorithm for moving/resizing floating panes on SIGWINCH? This is one of those "happens a lot in terminals but not that much in window managers" things and I want to nail the UX here. So far I'm going with "shrink size until minimum pane size and then start moving left/up" but that's not great when re-expanding the window (the panes stay in their moved positions), and I'd really like to avoid having to keep track of previous resizes.

[dankamongmen]

Amen to that :) By composition model do you mean regarding pane/window stacking and deciding what we render to the client? Since I'm just developing floating windows right now I'm just about to reach the point where I need to make this decision. My current gut feeling says it would be best to use a top-down approach for now, but I have not yet had the chance to devote much thought to the potential downsides. More reading material about said stacking problem would be greatly appreciated!

https://github.com/dankamongmen/notcurses/issues/1068

i have not yet fully investigated how general of a problem this is. it might be entirely an implementation detail of my rasterizer; it might be an issue all rasterizers must explicitly deal with. i suspect it afflicts all top-down rasterizers, but only them.

essentially, imagine you're composing two windows. one has a white UPPER HALF, and a transparent background. below it is a LOWER HALF, this one a blue foreground and a green background. the result should logically be a cell split in half vertically, with the top white and the bottom blue. unless you're very careful, though, you're likely to end up with the top white, and the bottom green.

[dankamongmen]

Also, while we're at this: could you recommend a good algorithm for moving/resizing floating panes on SIGWINCH? This is one of those "happens a lot in terminals but not that much in window managers" things and I want to nail the UX here. So far I'm going with "shrink size until minimum pane size and then start moving left/up" but that's not great when re-expanding the window (the panes stay in their moved positions), and I'd really like to avoid having to keep track of previous resizes.

i've concluded that there's no general answer to this problem. i instead allow each window to have a resizecb. a terminal resize invokes the resizecb of each "root window" (a window whose parent is itself), and whenever a window's geometry changes, the resizecbs of its children are invoked. this obviously can lead to a cascade of callbacks that touch every window on a terminal resize. i furthermore provide several simple resizecbs that users can select, beyond writing their own. they're documented here: https://notcurses.com/notcurses_plane.3.html

int ncplane_resize_realign(struct ncplane* n);
int ncplane_resize_maximize(struct ncplane* n);
int ncplane_resize_marginalized(struct ncplane* n);
int ncplane_resize_placewithin(struct ncplane* n);

beyond that, i strongly encourage you not to update anything on SIGWINCH yourself; make it available to the user, and let them act on it if they want. then, in your rasterizer, do a geometry check regardless of whether you saw SIGWINCH, since signals are hot garbage. at this point, begin a callback cascade (if you adopt that strategy), since the user has presumably loaned you some execution context (unless you're doing something asynchronous, in which case this is all inapplicable).

[dankamongmen]

i've concluded that there's no general answer to this problem. i instead allow each window to have a resizecb. a terminal resize invokes the resizecb of each "root window" (a window whose parent is itself), and whenever a window's geometry changes, the resizecbs of its children are invoked. this obviously can lead to a cascade of callbacks that touch every window on a terminal resize. i furthermore provide several simple resizecbs that users can select, beyond writing their own. they're documented here: https://notcurses.com/notcurses_plane.3.html

let one thousand flowers bloom, but let me warn you that these details are pretty unrewarding, and getting them right on every terminal is even less rewarding, and you might want to focus on higher-level stuff and use libnotcurses-rs as your lower-level backend. just a thought. it is of course my project, so take my recommendation with a grain of salt.

[imsnif]

I think my main issue with all of this is that 99% of the time these windows are terminal panes opened by a user of our application, who generally does not use code in order to interact with Zellij. Their flow is (or will be in this case):

1. press alt-n - a new floating terminal pane appears
2. resize terminal window - expect things to do what they want

Alas, not much room for a callback. Unless I misunderstood you? Might be good to allow a finer grained behaviour as part of our layout system though.

As for SIGWINCH - I must admit we've been using it for quite a while to detect window size changes (and trigger a re-render) and it's been working great across various terminal emulators. We've not had one issue with it (that I'm aware of). What issues have you experienced with it?

About detecting the geometry each time instead of relying on SIGWINCH: wouldn't that be very detrimental to performance? Also, how do we know to trigger a render? Or actually, I guess this is also a difference that comes from Zellij users not using it as a library inside their programs but rather as an app directly?

[ghost]

I'm also curious if one of you has special approaches regarding output buffers? Specifically, say I have a complex stack of panes on screen and a few characters change in one of them. Some of those characters are visible, some are covered by another pane. Do you have a clever way of rendering just those changed visible characters to the terminal? Or just the usual "loop through cells, query all panes with greater z-index to see if anything is covering them" approach?

No and yes. :)

Building up the logical screen is "loop through windows somehow, and end up with what the monitor should look like". That part will be quite different from top-down vs bottom-up. So that's the no part.

But once you have that logical screen, you want to sync it to the physical screen with as few changes as possible. For that part, here is non-image portion of the screen update in ECMA48Terminal.flushLine() . For each line, I compute a bounding box of text that has changed (the lastX / textEnd stuff), and then within that region place the cursor in front of each section of cells to redraw, and draw only the changed attributes (lastAttr) and characters. There is a little more optimization possible, but it's not too far off the minimum for the idea of replacing all different cells. But if one knew that the only real change was that a rectangular region(s) needed to scroll, but that was it, then you could just issue the scroll(s) and be done. Alas, when you enter cascaded windows land finding regions that would scroll or be copied around vs just replacing all of the different cells becomes a Harder Problem. And that's the yes part. :)

Also, while we're at this: could you recommend a good algorithm for moving/resizing floating panes on SIGWINCH? This is one of those "happens a lot in terminals but not that much in window managers" things and I want to nail the UX here. So far I'm going with "shrink size until minimum pane size and then start moving left/up" but that's not great when re-expanding the window (the panes stay in their moved positions), and I'd really like to avoid having to keep track of previous resizes.

twin, Turbo Vision (Sergio Sigala's version and @magiblot 's version ), and vtm are the other text-based WM's that have cascaded windows and actual users (unlike me). They might have some inspiration for you.

For me, I don't do anything special with SIGWINCH for cascaded, other than resize the entire desktop screen. I do have a drop-down menu available that can always be used to get windows that were resized off-screen to show:

One can select "Tile" or "Cascade" to get everything to reappear, even if it isn't quite where you wanted it to go.

Widgets (terminals) on the "desktop" are made aware of the screen resize though, and TSplitPane which is the container for terminals/splits will move the divider such that terminals always stay on screen -- they shrink but don't get bigger. For XtermWM's tiled terminals, it looks like this:

https://user-images.githubusercontent.com/4357501/150162727-e4834bb2-30bb-48f2-9423-392ce08eb2d9.mp4

Kind of ugly. One could also do something like StretchLayoutManager where the original positions are remembered, and everything scales: pics here.

[ghost]

and use libnotcurses-rs as your lower-level backend. just a thought. it is of course my project, so take my recommendation with a grain of salt.

For the record, if I ever start a new non-Java terminal project I will begin with notcurses. It has a very active developer and community, bindings to several languages, cross-platform support, and ludicrous speed. ;-)

[imsnif]

Thanks @klamonte - this is the general direction of what I was thinking about and it's good to hear about your implementation.

For tiling, we keep track of pane sizes as percentages and use a constraint system to solve them: https://github.com/zellij-org/zellij/blob/main/zellij-server/src/tab/pane_resizer.rs#L214 (if you're curious). Only issue is the rounding is sometimes inconsistent (but that's a relatively easy fix we need to get to at some point).

Once again - really enjoyed the discussion @klamonte and @dankamongmen! Thanks for it. Please drop by as often as you like. I imagine I'll be pinging both of you once I (finally!) get to implementing image support.

[AutumnMeowMeow]

"subscribe" (from my new account :) )

[imsnif]

Here's the rendering algorithm I came up with:

1. Gather all "changed chunks" (consisting of x/y coordinates and the changed characters) on screen
2. For each chunk, loop through all panes above it by z-index and "cut out" the parts of it that are hidden by the pane
3. Serialize what's left into VTE.

This is pretty fast performance-wise by my tests. Would you consider this painter's or reverse painter's?

[AutumnMeowMeow]

It sounds a bit closer to "reverse painter's", but with a different ordering of the loop. I'd assume each changed chunk has its own range of cells which was rendered separately, and then you are composing the final screen out of the previous screen and these deltas, accounting for the overlaps so it's like a jigsaw puzzle. gui.cs talks about computing optimal damage regions, this might be close to that.

[imsnif]

We're not keeping track of the previous screen per-se, since Zellij keeps track of the state of the screen anyway - so we use that. Every time a cell is changed, we mark it as changed and then on render we gather all these changes into chunks. But I guess it's mostly the same thing, yeah.

Is there any benefit to looping the other way around? I did it this way because I assumed in general it would be more common for larger floating panes to be on the bottom with smaller panes covering them - but I'm not sure how true (or how big of an optimization) this is...

[AutumnMeowMeow]

The main benefit is cells that have partial transparency: to get the screen right, you need more than one at the same x,y location to be rendered bottom-up. If you have no transparent/translucent cells, then you won't need to change anything.

[imsnif]

Ah, cool. I think when we implement transparent cells this won't require a lot of changing, since we "cut away" at the chunks starting bottom up. So instead of cutting away we could adjust their background as needed. If I got the method you described in your blog right.

[imsnif]

Also btw, the terminal-window resize algorithm I came up with (cc @dankamongmen )

1. We keep track of floating panes' "intended position/size", meaning either their initial position/size or something explicitly initiated by the user (eg. moving/resizing)
2. We attempt to resize/move them back to this position when there's room and not farther.

Requires keeping a little extra state, but I found this to have some satisfying predictability and a high "do what I want" factor. Hopefully other users will agree.

[imsnif]

This has been implemented in 0.25.0

[dmgolembiowski]

@imsnif this feature is sooooooo sick; thank you so much!

After working on this feature, what's the most straightforward way to make a hovering window stay in position after clicking off of it? There are some sweet watch-style CLI's like bacon that really benefit from having a dedicated window open, that constantly run cargo-check in the foreground. I know it's been a minute, but do you remember seeing files responsible for interacting between the floating pane and the ones beneath it? I think it would be fantastic if, after running Ctrl + p + w , the window pane awaits some keystroke (or click-combo) to make the floating window's foreground position more permanent.