use DECFRA to update glyphs underneath transparent graphic

[dankamongmen]

DEC420+ supports DECFRA (CSI Pc ; Pt ; Pl ; Pb ; Pr $ x):

CSI Pc ; Pt ; Pl ; Pb ; Pr $ x
          Fill Rectangular Area (DECFRA), VT420 and up.
            Pc is the character to use.
            Pt ; Pl ; Pb ; Pr denotes the rectangle.

see #1619 for why this might be expected to work better than P2=0. there appears to be no terminfo capability corresponding to this escape sequence, alas.

also, what the hell is defbi "define_bit_image_region" as mentioned in terminfo(5)?

[dankamongmen]

there's also DECERA

CSI Pt ; Pl ; Pb ; Pr $ z
          Erase Rectangular Area (DECERA), VT400 and up.
            Pt ; Pl ; Pb ; Pr denotes the rectangle.

[dankamongmen]

CSI Pt ; Pl ; Pb ; Pr $ {
          Selective Erase Rectangular Area (DECSERA), VT400 and up.
            Pt ; Pl ; Pb ; Pr denotes the rectangle.

when you wish to selectively nuke from orbit

[dankamongmen]

CSI Pt ; Pl ; Pb ; Pr $ {
          Selective Erase Rectangular Area (DECSERA), VT400 and up.
            Pt ; Pl ; Pb ; Pr denotes the rectangle.

when you wish to selectively nuke from orbit

it looks like this plays with DECSCA (select character protection attribute)

[christianparpart]

Most interesting i still think is DECCRA. That should also help improve performance and latency. It could be used to rerender moved parts on the screen in a single VT sequence.

[dankamongmen]

oooh DECCRA looks useful indeed, assuming it's actually widely implemented. with that said, it would be difficult for me to use it in my current model, except for sprixels without background glyphs, since everything else gets mixed down to a single layer during rendering. i could theoretically tag plane movements, and watch for interactions with other planes, and use DECCRA if there are none of the latter, but that's a fair amount of effort to avoid some glyph writes, especially if support is thin. for sixel, it could be useful indeed.

@ThomasDickey's observation here points out that styling/colors aren't copied over, which indeed looks to be the case:

The copied text takes on the line attributes of the destination area.

which is definitely the final nail in the coffin in terms of it being useful to me. the vast majority of non-bitmap bandwidth is going to styling escapes, not glyphs, so this wouldn't actually save me much bandwidth. and who knows whether it works on sixels?

nuking rectangular regions from orbit remains desirable.

[dankamongmen]

@dknl has some useful comments back in #1619:

It's hard to say though. Erasing a large sixel line-by-line can be really slow too, since the sixel may have to be copied and truncated over and over again. You definitely want to use escape sequences that erase a swat of cells in one go, rather than, say, printing spaces. There are escapes to do "rectangular editing" (and among them, a rectangular erase). Unfortunately they're not widely supported. Technically, they're only available on VT400 and newer (many terminals don't implement 400 features, or any 300 features for that matter). But, "rectangular editing" has it's own feature number in the primary DA response: 28. So, what I'm thinking is you can look for 28 in the DA response, ignore the VT level, and use DECERA (rectangular erase) if available. And fall back to ECH, ED, EL or whatever.... except that XTerm includes 28 in its DA response even in e.g VT220 mode, where it doesn't support rectangular erase. Unsure if this is a bug or if it's intentional.

his analysis of performance (and why P2=0 would likely be a miss) corresponds to a reply from @ThomasDickey a few weeks back when i reported what i thought a bug in XTerm performance.

[christianparpart]

@ThomasDickey's observation here points out that styling/colors aren't copied over, which indeed looks to be the case:

He is wrong as much as the spec is concerned at least. Quote from the spec: "The copied text retains its character values and attributes."

There may have been buggy implementations though. - or at least it differs from my interpretation of the spec. See below.

The copied text takes on the line attributes of the destination area.

which is definitely the final nail in the coffin in terms of it being useful to me.

Check https://vt100.net/docs/tp83/appendixb.html and search for "line attributes". Line attributes are those that apply to the whole line and not those in the cell level (such as SGR attributes) unless i am completely drunk (nothing is impossible). ;-)

I hope this text is free of topys as i am writing from the phone :)

P.s. VT >= 400 for those who care

[dankamongmen]

perhaps! your reading seems reasonable, but i am still no expert in such things.

either way, rectangular copy just seems very difficult to fit to my implementation, except perhaps for bitmaps (when are rasterized as a plane, as opposed to glyphs, which are all rendered down to a single common frame first). when it comes to bitmaps, with Kitty i've got true moves, and with any new graphics protocol, i would hope for the same (and even with kitty's move, i can't use it very often due to other problems, see #1803 and #1395).

but i've got big hopes for DEC[EF]RA! need to experiment.

[dankamongmen]

looks like it might be sufficient to check for 28 in the Device Primary Attributes result.

[christianparpart]

looks like it might be sufficient to check for 28 in the Device Primary Attributes result.

Exactly, Mr.Robot.

[dankamongmen]

DECERA doesn't appear to erase sixels in at least XTerm =[

[dankamongmen]

ED (Erase Display) does seem to apply to sixels, but only gives us complete display, below cursor, or above cursor. EL (Erase Line) does seem to apply to sixels, but gives us only complete line, left of cursor, or right of cursor

[dankamongmen]

EA (Erase Area) would presumably be perfect, but doesn't appear implemented very widely =[

[dankamongmen]

ECH looks like it might work! it's not as good as DECERA would be, but it does appear to wipe out sixel, and do so from the cursor, and through a controllable length. yessssssssssssssssssssss

[dankamongmen]

though, i was thinking about this, and it's all irrelevant. we don't go printing a ton of spaces atop sixels to erase them; we go printing whatever was in the damage buffer. otherwise, we'd just have to draw the latter for a total of two draws per cell.

• "but dank," we thought, "this would still work if the damage buffer just had a lot of clear cells. ought we microoptimize for that case?"
• "bah, other dank, who cares? that's not so frequent of a case. if we at least covered strings of every glyph, then, maybe, but this is just (uncolored) spaces"
• "but dank", we continued, "we can actually do that, right? with repeat character, from terminfo's rep"
• "don't be a maroon, other dank, we'd need identify such regions, and by then you might as well have just written them out"
• "at render time, dank, sure -- though the idea that saving a byte of output is multiplied in impact might say otherwise. but what about at raster time? all you need do is lookahead into data you're about to write if you don't find such a region. there's no extra cache misses, and the work will surely be hidden under the AMAT. then it's just emit once, and use CSI Ps b, Ps=N, which at 4 + logN bytes beats 6 reps or more, and stomps Sixels just fine. you can even use CSI Ps X, Ps=N to beat just 5 spaces, though i'd probably just want to rely on a single escape's support."
• "not too shabby, dank the third, and it makes us think--"
• "[danks 2, 3, and 4 in unison] if we did perform it at render time, we could use uplinks to find rectangular regions in O(1)"
• "danks, c'mon, remember? no rectangular replace char support atop sixels"

that original dank, such a buzzkill!

[christianparpart]

DECERA doesn't appear to erase sixels in at least XTerm =[

This sounds like a bug in xterm? Can it be fixed?

[dankamongmen]

DECERA doesn't appear to erase sixels in at least XTerm =[ This sounds like a bug in xterm? Can it be fixed?

does contour erase sixels with DECERA? i didn't test beyond XTerm. the documentation at vt100.net didn't seem to state explicitly one way or the other.

like i noted later, i don't need DECERA in any case, since i changed things up two months back or so to simply reprint the cells underneath the sixel, which annihilates it while putting up the necessary backglyphs. so it doesn't buy me anything. i could possibly make use of DECFRA in the future, but i can't yet. using it with spaces, if those knocked out the sixel, would seem functionally equivalent to DECERA, so i'd just use that for that case.

let me go take a look at the XTerm code surrounding DECFRA and see if there are any indications...

[dankamongmen]

if it was going to happen, it would presumably happen in ScrnWipeRectangle(). ScrnWriteText() calls chararea_clear_displayed_graphics(), let me see if tucking the latter into the foremost solves this problem. if so, i'll send it off as a patch to Grandmaster Dickey, and we'll see what happens.

[dankamongmen]

ScreenFillRectangle(), rather

[dankamongmen]

sweet, my patch worked immediately. i'll send it on out.

[christianparpart]

does contour erase sixels with DECERA? i didn't test beyond XTerm. the documentation at vt100.net didn't seem to state explicitly one way or the other.

Of course it does (in contour). ED is erssing it all too without mentioning Sixel, doesn't it.

Also, i am not sure the spec mentions that char writes are replacing image cells do they?

[dankamongmen]

xterm-368-decerasixels.diff.txt

godspeed little diff

[christianparpart]

xterm-368-decerasixels.diff.txt

godspeed little diff

Are you living with a cable plugged into the powerplant?

[dankamongmen]

Date: Sat, 26 Jun 2021 02:55:24 -0400
From: nick black dankamongmen@gmail.com
To: dickey@invisible-island.net
Subject: [PATCH] apply DECERA/DECFRA to sixels
Bcc: notcurses@googlegroups.com

[-- Attachment #1 --]
[-- Type: text/plain, Encoding: 7bit, Size: 1.6K --]

Grandmaster Dickey,

I have found that DECERA/DECFRA do not apply to Sixel within the
specified rectangular region. The documentation I've read
doesn't seem explicit one way or the other:

https://vt100.net/docs/vt510-rm/DECERA.html
https://vt100.net/docs/vt510-rm/DECFRA.html

but perhaps you have documentation beyond this that locks things
down. Alternatively, perhaps the six-row nature of sixel isn't
immediately amenable to rectangular erasure, since it might end
mid-cell? But the possibility remains that this was merely an
oversight, in which case the attached patch ought at least be a
starting point for remedying the issue.

This probably ought apply to DECSERA, DECSEL, etc. If so, the
attached patch ought be easily enough generalized.

It seems to me intuitive and expected that these control sequences
would obliterate all (non-protected) content. Erase Display,
Erase Character, and Erase Line do (by test and visual
inspection) apply to Sixel, which seems correct (I've been
unable to get Erase Area working in any capacity, and assume
it's an aixterm-only deal).

This came up in the following Notcurses issue, if you're interested:

https://github.com/dankamongmen/notcurses/issues/1740

Please feel free to use this patch however you see fit, or to
reject it if inappropriate.

BTW, thanks for fixing up the TIOCGWINSZ-on-font-change issue in
Patch 368! I updated the historical reference at

https://nick-black.com/dankwiki/index.php?title=Theory_and_Practice_of_Sprixels#Other_people.27s_bugs

Hack on!

--nick

[dankamongmen]

Are you living with a cable plugged into the powerplant?

bah, patching well-written c is easy. now enriching uranium to weapons-grade using TEA carbon lasers, some Raman filters, and reverse-engineered SILEX, that takes talent.

[dankamongmen]

now that i think about it, it might be good to keep the current behavior. like i said, i have a way to wipe out sixels that works just fine, with only one write per cell. what i don't have is a way to update the glyphs under a partially-transparent sixel cell (sprixcell) without repainting the whole goddamn thing. if DECFRA works under transparent sixels, that could be very advantageous!!

tested and IT DOES WORK. holy shit, this could be a major win!

[dankamongmen]

[dankamongmen]

oh this is very very exciting indeed, hot shit goddamn. alright, it is imperative that that patch not be applied, and that the current behavior be preserved. i can pick up a big win from this.

[dankamongmen]

Date: Sat, 26 Jun 2021 03:56:28 -0400
From: nick black dankamongmen@gmail.com
To: dickey@invisible-island.net
Subject: Re: [PATCH] apply DECERA/DECFRA to sixels

nick black left as an exercise for the reader:

Grandmaster Dickey,
I have found that DECERA/DECFRA do not apply to Sixel within the
specified rectangular region. The documentation I've read
doesn't seem explicit one way or the other:

Please withdraw this patch and suggestion, at least for DECFRA.
I just discovered that DECFRA replaces glyphs underneath a
partially-transparent Sixel without damaging the Sixel, which
ought allow me a tremendous speedup when I need perform such an
update.

Currently, if I have a glyph underneath a partially-transparent
Sixel, and I want to update it, I have to do so, and then redraw
at least the damaged part of the Sixel. Since "redrawing" such a
partially-transparent Sixel means first touching every glyph it
covers, I'm effectively drawing the entire affected region twice
to change any single glyph underneath it. With DECFRA working as
it currently does, I can just emit those as single-cell DECFRAs,
and avoid the entire redraw. That's great. That's a huge win.

--
nick black -=- https://www.nick-black.com
to make an apple pie from scratch,
you need first invent a universe.

[christianparpart]

Now i am feeling confused about what the exact semantics of the VT sequences with respect to Sixel graphics is.

If ICH is splitting the sixel vertically in half then DECERA should also erase affecting cells respectively. Analogous all the oth r VT sequences.

[dankamongmen]

so, coming back 'round to this and summarizing the above: tl;dr

DECFRA allows us to replace a region of glyphs with some other glyph. on at least XTerm, it honors the current color and styling. most importantly, it works underneath a transparent graphic without redrawing the entirety of said graphic.

i'm not sure if any other terminals support it, or if it can be discovered via anything more specific than primary DA.

• MLTerm: supports it, but obliterates graphics =[
• kitty: no support, no need
• alacritty: no support
• WezTerm: no support
• contour; no support
• foot: no support

so this would only be of much use with XTerm, it seems. but XTerm is where we have our worst performance, so maybe it's worth the one-off. nothing high-priority, though.

[dankamongmen]

whether or not DECERA applies to sixel is the 64,000 question in https://github.com/hackerb9/vt340test/issues/16

[dankamongmen]

hey @dnkl , do you intend to support rectangular editing operations in foot? they're not being advertised at the moment.

[dnkl]

@dankamongmen no intentions, up to this point at least. Simply because I'm not aware of anyone using them. If notcurses starts using them, I'll reconsider.

(assuming there's consensus on how they should interact with sixels...)

[christianparpart]

tmux can do, IIRC. I noticed that because I initially implemented them buggy.

On the other hand are we currently talking about a chicken'and'egg problem. I think they are doing good, so I think they deserve to be implemented.

(I am going to verify my claim as soon as I. At a computer).

Cheers ;-)

[dnkl]

tmux can do, IIRC

If I'm not mistaken, tmux only emits them when running on a VT420 level emulator, or newer (instead of e.g. checking for "Rectangular Editing" in the primary DA respone. If so, then adding only these escapes to foot wont be enough, since there are many other unsupported VT420 escapes.

That said, I think we can still add them, if e.g. notcurses wants them.

[dnkl]

@dankamongmen btw, I tried your ncplayer + DECFRA example (from https://github.com/dankamongmen/notcurses/issues/1740#issuecomment-868964356) in XTerm 370, and it seems it now erases the sixel:

                       Patch #370 - 2021/11/13
...
* modify  DECERA  and  DECFRA  to  erase  corresponding area in SIXEL
  graphics  (patch  by  Nick  Black, notcurses #1740, vt340test #16).
  Also modify DECSERA.

[dankamongmen]

• modify DECERA and DECFRA to erase corresponding area in SIXEL graphics (patch by Nick Black, notcurses #1740, vt340test #16). Also modify DECSERA.

yep =] i'm nick black =]

[dnkl]

yep =] i'm nick black =]

🤣 aye, I'm aware of that. But:

alright, it is imperative that that patch not be applied, and that the current behavior be preserved. i can pick up a big win from this.

... and I didn't see a conclusion being reached in https://github.com/hackerb9/vt340test/issues/16.

But, having DECERA and DECFRA erase sixels makes sense, and is consistent with other similar escapes.

I'll get started on an implementation for foot.

[christianparpart]

Nice @dnkl , looking forward to it. (I also implement them and expose it also regardless of VT identification).

WRT tmux we could file a ticket to also respect rectangular feature exposure through DA1 in tmux.

[dnkl]

Re. DECCRA; I agree with @christianparpart that, at least according to the spec, SGR attributes are copied as well, along with the characters.

[dnkl]

I'll get started on an implementation for foot.

https://codeberg.org/dnkl/foot/pulls/864

[dnkl]

WRT tmux we could file a ticket to also respect rectangular feature exposure through DA1 in tmux.

@christianparpart tmux recognizes the non-standard terminfo capability Rect. When present, it will use DECFRA even if the DA responses don't indicate VT420 level support.

Note that it's a string capability (and not a boolean, as I first thought). See https://codeberg.org/dnkl/foot/commit/375441a99bb8a111c8a21d120e447063e49b34b9

[christianparpart]

@christianparpart tmux recognizes the non-standard terminfo capability Rect. When present, it will use DECFRA even if the DA responses don't indicate VT420 level support.

Note that it's a string capability (and not a boolean, as I first thought). See https://codeberg.org/dnkl/foot/commit/375441a99bb8a111c8a21d120e447063e49b34b9

Why does tmux keep inventing new terminfo entries. @dnkl please also advertise via DA1 at least. What you ultimatively want is:

• DECFRA - fill area
• DECERA - erase area
• DECCRA - copy area

The ones I have not yet implemented but do consider useful (i.e. will do soonish):

• DECCARA - change attribs
• DECRARA - reverse attribs (not all, checks vt100.net)

IMHO useless ones (subjective):

• DECRQCRA - request checksum of area (nice, but honestly I don't think there's any use)

You can find them all documented at: https://vt100.net/docs/vt510-rm/contents.html

I think I should probably add Rect to my terminfo though, too. Thanks for the pointer @dnkl :)

[dankamongmen]

i am currently only looking for 28 in DA1. i'm trying to get away from terminfo, so i might pick up Rect, might not; let me marinate on it for a minute.

[dnkl]

I mentioned the Rect terminfo capability as an alternative to trying to convince tmux to pick up 28 from DA1 (or as an interim solution until they do...).

Obviously, advertising via DA1 is better.

The foot PR linked above implements:

• 28 in DA1
• DECCARA
• DECRARA
• DECCRA
• DECFRA
• DECERA

[dnkl]

Why does tmux keep inventing new terminfo entries

Some of them make sense, imho. For example, the setrgbb and setrgbf (in combination with the Tc flag), as an alternative to having a separate <term>-direct terminfo file.

[christianparpart]

@dankamongmen please don't support Rect, it's wrecked.

[dankamongmen]

but is it the Tc or the RGB capability? terminfo claims the latter.

i was unaware of setrgb{bf}, sweet

[dnkl]

@dankamongmen RGB tells you the terminfo is a *-direct variant, and that the regular setab/setaf capabilities can be used to set 24-bit colors.

Tc on the other hand (another tmux addition) tells you the terminal is truecolor capable, and that the regular setaf/setab capabilities are used to set colors from the 256-color palette. And that the new setrgbf/setrgbb capabilities can be used to set 24-bit colors.

(I guess technically the Tc flag isn't needed - you can just check for setrgbf/setrgbb. Or, I've missed something).