tmux compatibility

[crabdancing]

So, I asked the tmux people if they would be willing to add even a marginal amount of support for libsixel, and they pretty much dismissed it out of hand. So now I'm thinking about forking tmux to add SIXEL compatibility. Is there anyone here who could help with that?

[cipri-tom]

so, iTerm2 has an utility called imgcat which can also display images inside the iTerm2 terminal.

This was wrapped to support tmux, and it can be seen here: https://iterm2.com/utilities/imgcat

Maybe it provides a starting point ?

[xaizek]

Based on comment in imgcat script, this seems to pass the image through tmux:

printf "\033Ptmux;\033\033]"; img2sixel test.png | sed 's/\x1b/\x1b\x1b/g'; printf "\a\033\\"

Given that there is a -P option to do the same for GNU Screen, it might make sense to add support for tmux to -P as well (autodetect terminal multiplexer via environment variables and allow optional argument to explicitly specify kind of escaping).

[ghost]

I just added sixel support to my Jexer TUI toolkit's terminal window. (The xterm backend got sixel output earlier this year.) The hard parts of this work are:

1. You must decode the sixel data coming from the "inner" or remote terminal into bitmap, and break that bitmap up into text cell-by-text cell image data.
2. You must know how the windows overlap so that only those visible cells are rendered, and then make some other tricks to actually mix the text and image data.
3. Once you know which image data needs to be rendered, you have to re-encode the bitmap portions into sixel.
4. tmux has the added wrinkle that they must do this against ncurses -- many different versions of ncurses -- that has its own strategies for managing the screen. It is difficult to battle what ncurses thinks is on the screen vs what you think should be on the screen when you are going outside ncurses' supported features. But they've done it before with 24-bit RGB, so this might be too hard for them.

I have managed it, but the performance is nowhere near libsixel's. (I will be spending some time later trying to get my sixel encoder better.) But I mention it because the code may be useful for others to work with (it is licensed MIT), and the concept itself can be played with. Also, so far I have only encountered three terminals that can actually mix sixel-and-text correctly: xterm of course, yaft, and now Jexer. EDIT: Also mlterm since version 3.8.8.

Finally, xterm's images support is a bit buggy: if you put an image on, and then overwrite part of that image with text, then the space to the right and below the cell you changed gets corrupted. tmux may need to do the same trick I did: emit all of the text cells, then overwrite all of the image data on the rows that have changed. For performance on xterm's side, one should try to combine adjacent image cells into contiguous images.

[nicm]

tmux has the added wrinkle that they must do this against ncurses -- many different versions of ncurses -- that has its own strategies for managing the screen.

tmux does not use ncurses except for reading terminfo.

[ghost]

tmux has the added wrinkle that they must do this against ncurses -- many different versions of ncurses -- that has its own strategies for managing the screen.

tmux does not use ncurses except for reading terminfo.

Thanks for the correction. I had assumed you were using ncurses for screen since I have seen several tmux issue threads that refer to ncurses versions.

Also another correction I need to make:

Finally, xterm's images support is a bit buggy: if you put an image on, and then overwrite part of that image with text, then the space to the right and below the cell you changed gets corrupted.

I saw this behavior last year with xterm, but no longer see it quite the same on xterm 344. On my last check, if you put a sixel image on screen, and then delete cells in the middle of it, then the visible screen shows the image as shifted over on that line as though the pixels are tied to the cells. However, if you then select the area with the mouse, the full image shows up again. So the image is not really broken up into cells, but can look that way depending on how one sequences the insert/update/delete operations.

[nicm]

Yes, the ncurses version only really matters inasmuch as it is also the version of the bundled terminfo database. I'm not really surprised you are hitting sixel bugs since you are probably by far the most sophisticated user of sixel.