deal with alacritty's weird sixel erasure semantics

[dankamongmen]

Running the bitmapstates PoC in various terminals, we see some problems. IMHO we ought get these resolved for 2.4.0, probably pushing it back. =[

kitty: the last three states seem wrong

• "full square on right" is empty square on right
• "ought see 16*s" empty square below and to right, very off
• "full square on right" is once again an empty square

alacritty: 2 and 4 plus one later

• "ought see 16 *s" is full square
• "ought see empty square" is full square
• "ought see 16 s" is full square but "ought still see 16 s" is correct

xterm: perfect, nice work xterm

contour: two breakages:

• "ought still sett 16*s" is empty square

i haven't been able to build wezterm in a bit =(

[dankamongmen]

looking into alacritty, on the "ought see 16 *s" we move from

Sprixel 9127379 (0x5610970cea20) 176B 6x6 (120x60) @0/0 state: 1
000000                                                                                                                       
000000                                                                                                                       
000000                                                                                                                       
000000                                                                                                                       
000000                                                                                                                       
000000 

to

Sprixel 9127379 (0x5610970cea20) 176B 6x6 (120x60) @0/0 state: 0                                                             
000000                                                                                                                       
055550                                                                                                                       
055550                                                                                                                       
055550                                                                                                                       
055550                                                                                                                       
000000

so one, we went from UNSEEN to QUIESCENT on the sixel overall, though the sprixcells transitions are correct (SPRIXCELL_OPAQUE_SIXEL to SPRIXCELL_ANNIHILATED). it definitely ought not be QUIESCENT.

[dankamongmen]

this does indeed fix alacritty. xterm remains correct. butttttttttttttt xterm now leaves copies of the orca around, and alacritty's copy problem is exacerbated. xterm's intro does look a bit cleaner, though....but that's not worth it at the expense of duplication =[.

[dankamongmen]

ahhh, properly using sprixel_invalidate() rather than a pure transition eliminates the duplication in xterm, huzzah! committed; alacritty is now perfect on bitmapstates.

[dankamongmen]

though this raises the question...why was it working in xterm, if we weren't invalidating?

which raises another question: why would we need invalidate on a wipe of sixel? that just means we're going to print a character over it, destroying the affected area. we shouldn't need redraw it. hrmmmm...

was alacritty not printing over the graphics? let's take a closer look...

[dankamongmen]

now regarding kitty: things work perfectly on an older version. so presumably we're doing something wrong with animation.

[dankamongmen]

so exposing RAST in alacritty, we do seem to be knocking things out:

notcurses_rasterize_inner:1184:pile 0x55c9829070a0 ymax: 70 xmax: 115
notcurses_rasterize_inner:1194:Sprixel phase 1
clean_sprixels:822:Phase 1 sprixel 9127379 state 0
notcurses_rasterize_inner:1199:Glyph phase 1
RAST 00000031 [1] to 6/10 cols: 1 0000000000000000
RAST 00000036 [6] to 6/11 cols: 1 0000000000000000
RAST 00000020 [ ] to 6/12 cols: 1 0000000000000000
RAST 0000002a [*] to 6/13 cols: 1 0000000000000000
RAST 00000073 [s] to 6/14 cols: 1 0000000000000000
RAST 00000000 [] to 6/15 cols: 0 0000000000000000
RAST 00000000 [] to 6/16 cols: 0 0000000000000000
RAST 00000000 [] to 6/17 cols: 0 0000000000000000
RAST 00000000 [] to 6/18 cols: 0 0000000000000000
RAST 00000000 [] to 6/19 cols: 0 0000000000000000
RAST 00000000 [] to 6/20 cols: 0 0000000000000000
notcurses_rasterize_inner:1203:Sprixel phase 2
notcurses_rasterize_inner:1212:Glyph phase 2
RAST 0000002a [*] to 1/1 cols: 1 0000000000000000
RAST 0000002a [*] to 1/2 cols: 1 0000000000000000
RAST 0000002a [*] to 1/3 cols: 1 0000000000000000
RAST 0000002a [*] to 1/4 cols: 1 0000000000000000
RAST 0000002a [*] to 2/1 cols: 1 0000000000000000
RAST 0000002a [*] to 2/2 cols: 1 0000000000000000
RAST 0000002a [*] to 2/3 cols: 1 0000000000000000
RAST 0000002a [*] to 2/4 cols: 1 0000000000000000
RAST 0000002a [*] to 3/1 cols: 1 0000000000000000
RAST 0000002a [*] to 3/2 cols: 1 0000000000000000
RAST 0000002a [*] to 3/3 cols: 1 0000000000000000
RAST 0000002a [*] to 3/4 cols: 1 0000000000000000
RAST 0000002a [*] to 4/1 cols: 1 0000000000000000
RAST 0000002a [*] to 4/2 cols: 1 0000000000000000
RAST 0000002a [*] to 4/3 cols: 1 0000000000000000
RAST 0000002a [*] to 4/4 cols: 1 0000000000000000

[dankamongmen]

....yes, that seems to be the case, wtf. in alacritty, it appears that drawing a glyph over a graphic does not necessarily destroy it. wtf?!?

wait. we're drawing default glyphs on a white box. is it possible that glyphs print with transparency, and it's just being hidden!?

[dankamongmen]

nope, changing the color up doesn't help. BUT, this explains why alacritty is duplicating the orca in intro. though at some point, it cuts off...very strange. so it does work in alacritty if we have the sprixel invalidation, which triggers a redraw. are transparencies in graphics effective in alacritty sixel?

MADNESS.

[dankamongmen]

in #1740 we found that DECERA doesn't work on alacritty, so we can't help with that. ugh.

yeah, from all i can tell, we can't obliterate this sixel with glyphs in alacritty. what on earth?

[dankamongmen]

yeah. load a full-screen sixel with ncplayer -bpixel ../data/worldmap.png -k. you can't kill this with glyphs. wtf. how does anything work on alacritty? keller wipes things out just fine. auhgughughguhgughguhg

[dankamongmen]

it seems that if i start above a sixel in alacritty, and hit enter until i come out the bottom, that erases it. so maybe alacritty is an all-or-nothing deal, where you have to touch each line? i mean, that's bizarre, but that's what it's looking like....

...and that would match the behavior we're seeing. keller draws a fullscreen cell blit, which hits every line of the sixel. in intro, we duplicate the orca up until the front imago hits the HUD. at that point, duplication ceases. hrmm if that's true, we'd expect that with no HUD/FPS graph, we'd get complete duplication, let's see...nope, that's not what's up there (and we hit the FPS graph almost immediately anyway, bad theory). but keller is explained with this theory. hrmmm, let's see if bitmapstates works without the invalidation if we do [0..5] instead of [1..4]...yes, that does destroy the entire graphic!

ok, this is insane. maybe file a bug against @ayosec? either way, dealing with this will require some thought and work. let's move that to 3.0. retitling this bug.

[dankamongmen]

ahhh, and this explains why notcurses-input's plot looks correct in alacritty despite the missing glyphery (#2133). mystery solved!

[dankamongmen]

it looks like applying a new graphic atop the old graphic is also sufficient to destroy it.

so alacritty seems to be pursuing a semantics between kitty and sixel -- clearly a sixel has a holistic concept here. this is yet another place where mosaics could make things much simpler.

[dankamongmen]

see https://github.com/hackerb9/vt340test/issues/16 for questions about whether DECERA applies to sixel

[dankamongmen]

i think the best way to "solve" this would be to get upstream to change it, given that it still hasn't been merged into main alacritty.

[dankamongmen]

yeah if i could loop @ayosec in on this, that would be best

[dankamongmen]

i've described this at https://github.com/alacritty/alacritty/pull/4763#issuecomment-948112289

[dankamongmen]

the best way to deal with this might be mosaics. this is only an effect of the current implementation, not planned behavior, according to @ayosec. i definitely don't want to write a second raster path to deal with that if it might be going away (and hasn't yet been officially merged in any case). so maybe we just stop supporting alacritty until we have mosaics (we have special-case code to even recognize alacritty as a sixel provider today)?

[dankamongmen]

i'm disinclined to write code to deal with this. as noted above, mosaics makes it go away. who knows whether the graphics branch will ever get merged? its author sounds not disinclined to fix this there. yeah, i don't think we're going to special-case this in Notcurses. leaving open to verify that mosaics eliminate the problem down the road.

[ghost]

I think mosaics would indeed solve it for you. I have an old version of alacritty-ayosec and images look the same as all other terminals.

[dankamongmen]

I think mosaics would indeed solve it for you. I have an old version of alacritty-ayosec and images look the same as all other terminals.

oh, it looks the same when you draw it. the discrepancy happens when you try printing glyphs over it.

[ghost]

In a given line, I'm printing the images for that line, and then printing text over any cell that had an image before -- i.e. I am always destroying every image from the previous frame on that line, either with another image over it or text over it. That originally evolved out of seeing xterm screw image areas up that I hadn't touched at all on the latest frame.

[christianparpart]

Hey, I just found this ticket by accident (had time to looks around). Is contour still a problem here? What should I fix / improve? :)

[ayosec]

Is this still an issue in Alacritty with the latest changes?

[joseluis]

hi @ayosec I'm no expert on this issue but I just compiled your latest update on alacritty graphics branch v0.11.0, and run the bitmapstates PoC on it, comparing its output with xterm's.

Now it appears the output is all exactly the same as xterm's, except the "ought see empty square" is an empty square indeed, but the characters behind it don't show, like xterm does. Although I'm not sure the relevance of that:

[ayosec]

Now it appears the output is all exactly the same as xterm's,

Thanks for testing it!

except the "ought see empty square" is an empty square indeed, but the characters behind it don't show, like xterm does. Although I'm not sure the relevance of that:

This is something that I noticed when I was investigating how to implement the support for graphics in Alacritty. Xterm never replaces text, it just draw the graphic on top of it. I put some other examples in https://github.com/alacritty/alacritty/issues/910#issuecomment-613221752.

We can replicate that behaviour in Alacritty, but I don't know if it is a good idea. Does anybody know what the original DEC terminals did?

[christianparpart]

What are these "latest changes"? :)

On Fri, Jul 15, 2022, 3:24 PM Ayose Cazorla @.***> wrote:

Is this still an issue in Alacritty with the latest changes?

— Reply to this email directly, view it on GitHub https://github.com/dankamongmen/notcurses/issues/2142#issuecomment-1185543064, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAN3OZ32CW3EKQW3DMQYPTVUFRBLANCNFSM5DPGZIZA . You are receiving this because you commented.Message ID: @.***>

[ayosec]

What are these "latest changes"? :)

There is a summary in https://github.com/alacritty/alacritty/pull/4763#issuecomment-1184949754.

The tl;dr is that most problems described in the description of this issue should be fixed.