investigate mosaic performance on various terminals

[dankamongmen]

I'd like to begin the move to mosaics in 3.1.x. We first need to get a quantitative look at how this approach will perform in various terminals. Hand-construct a mosaic equivalent for a 10x10 (cell) and fullscreen image in both sixel and kitty, and see how the two compare. I imagine that mosaics will be a bit slower, but hopefully not more than 10% or so.

If the performance is bad enough, we'll need to revisit the idea. If it doesn't come back too bad, let's move forward with it. If it's particularly bad on some terminals, but fine on others, we'll....make a decision then.

[ghost]

Love it! 💖

If you tested today, I would be unsurprised if iTerm2 and DomTerm behaved oddly: I've seen recent screenshots with lots of artifacts -- seems like they either aren't keeping up with cursor movements when placing the image, or aren't destructively overwriting image with text. Don't bother with darktile yet. Doing lots of small images is currently tough on xterm.js + xterm-image-addon performance in the default configuration, but the screen itself is perfect; in an alternate config it's zippy as hell; jerch is doing his magic on it. For Kitty images, I saw differences in Kitty and WezTerm -- I've got an issue open with wez, but with only two doing it it's uncertain which is really correct as per the spec; they can sort it out and I will change my end if need be.

[dankamongmen]

if it comes back worse than hoped for, maybe we can go with fractured images, which of course reduce to mosaics in the limit (but don't let us eliminate all the in-image wiping and rebuilding code that makes mosaics so desirable). that's a lot of work just to get text atop images within a cell, though.

[dnkl]

"subscribe"

[AutumnMeowMeow]

"subscribe" (FYI @dnkl Hello from my new GH account. :) )

[dankamongmen]

so i think the way to start this is to select some sample images. let's take a few synthesized ones and a few more representative ones:

• a 200x200 image of all one color
• a 200x256 image of 256 colors in 200-pixel rows
• a 256x200 image of 256 colors in 200-pixel columns
• AllRGB's balloons.png
• data/notcurses.png
• data/chunli23.png

since it's difficult to force a given terminal emulator into a particular cell-pixel geometry, we'll need something that can carve the image up for the particular terminal graphic type {Sixel, Kitty} (linux framebuffer and other mmap interfaces have no concept of graphics, and thus this isn't relevant) and cell-pixel geometry. it ought be trivial enough to write up a C PoC to do this.

said PoC will do ncvisual_from_file(), render it as a single plane, and render it as the appropriate number of single-cell graphics. this will allow us to both time the Notcurses path, and capture the output to test without Notcurses (and get pure terminal numbers).

i don't see any method but visual inspection to ensure things continue to look correct.

this ought get us the necessary initial data to assess this scheme's feasibility. any Notcurses overhead delta can probably be cut down, but if the terminals themselves just puke on these inputs, we'll have to make some hard decisions.

[dankamongmen]

i should also repeat here that the sixel rebuild algorithm since #2603 is a real pile of shit, since i didn't want to invest much time if we were going to replace it with mosaics. if we don't, i should really go back and improve it. see #2603 for notes.