Question about VT340 SGR support and relations to SIXEL

[jerch]

For xtermjs I am thinking about a composite mode for layered SIXELs (and graphics in general, once we have something usable there). Per vaguely phrased spec SIXEL should support "composition" from anything thats on the screen in those 0-bit pixels, when the background param is set to transparent. Can you confirm that is the case on a VT340? Esp. for these cases:

• SIXEL with "holes" on colored background - background color shines through
• SIXEL with "holes" on foreground - parts of the foreground glyphs shine through
• SIXEL with "holes" on previous SIXEL - parts of the previous SIXEL is still visible

I can create a few simple test cases for that, if you want.

Another question is regarding SGR and transparency, as defined in ITU T.416 as SGR 38 ; 1. Does VT340 supports this (my guess would be no)? Because I was thinking about this as the opposite "SIXEL hole" case - when there is already graphics, text entered with SGR 38 ; 1 could blend on top of that graphic. (Still evaluating things here, not sure if I will go with that in the end, as its cumbersome to realize on renderer level.)

[hackerb9]

I know nothing about SGR 38. Do you have a test case?

Other than that, it works exactly as you described: 1-bit transparency on a framebuffer, same as GIF. Existing pixels — background color, text, and previous sixel images — "show through" transparent pixels.

If you'd like to create some simple test cases, I can send you a screenshot of what it looks like on my VT340. If you do, I recommend not defining any colors in your test.

You probably already know this, but, unlike modern sixel implementations, the VT340 reuses the color palette in a somewhat peculiar way so previous images, background color, and foreground color, can all change when a new sixel image is sent. This only occurs when colors are defined, but that is actually optional. You can just use the current palette.

If you do try to define colors, you should know that no matter what number is given to the sixel colors (e.g., #0), VT340 color registers are overridden in the order sixel defines them. The first one defined in a new sixel image always overwrites the first color defined in any previous sixel image, and so on. Additionally, the sixth color defined changes the foreground color, the 15th color changes the bold text foreground, and the 16th color changes the background.

For your simple test case, it would be best to avoid all that and just use the default palette.

[j4james]

You can see what transparent images look like when rendered over text, and also overlapping each other, in the test case here: https://github.com/hackerb9/vt340test/blob/main/j4james/image_overlap.png

And text on top of an image just erases the cells that are overwritten. I don't have a specific test case for that, because I thought it was well known behaviour, but you can see an example of it here (the --- text is rendered on top of the images): https://github.com/hackerb9/vt340test/blob/main/j4james/cursor_position.png

Another question is regarding SGR and transparency, as defined in ITU T.416 as SGR 38 ; 1.

The VT340 doesn't even support SGR color attributes, so I'm pretty sure it's not going to handle that. I'm not sure the T.416 standard even existed at the time the VT340 was developed did it? And in any event, it wasn't originally intended to be used by terminals as I understand it - I think it's only recently that it's been repurposed by modern TEs.

[jerch]

Thx for all your responses. Will look through the images to check whether they explain all my edge cases.

Tbh when I looked up the ECMA stuff again some time ago and tried to link it to actually devices of that time - well it seemed to me, that the whole ANSI/ECMA thing was mostly DECs work in the end (or at least the blueprint for the spec), as most other common vendors had their very own sequence styles plus some sort of ANSI/DEC compatible modes. I am not sure where T.416 comes from, it reads as for a much broader audience (general text repr systems). Not even sure if there was some sort of a predecessor already containing that SGR stuff.

[j4james]

the whole ANSI/ECMA thing was mostly DECs work in the end (or at least the blueprint for the spec), as most other common vendors had their very own sequence styles

I always assumed DEC were just one of the early implementers of the standard - possibly not even the first. Prior to ECMA-48 they also had their own sequences - remember the DEC VT52 used simple single character escape sequences (e.g. ESC A to move the cursor up). They only released the VT100 a couple of years after ECMA-48 was published AFAIK.

[jerch]

I always assumed DEC were just one of the early implementers of the standard - possibly not even the first. Prior to ECMA-48 they also had their own sequences - remember the DEC VT52 used simple single character escape sequences (e.g. ESC A to move the cursor up). They only released the VT100 a couple of years after ECMA-48 was published AFAIK.

Possible. Still other than other vendors, DEC kinda jumped into the ANSI boat as their genuine sequence format for VT100+ (even their privates follow the schematics). While for example Wyse kept strongly advertising their own formats for a long time (in some Wyse manuals the DEC/ANSI modes read like second class citizens). Well, thats only my observation from the manuals, it does not account for other strategic reasons like market competition and such.

[jerch]

I checked the example output and they indeed explain enough for my questions regarding text layering. The transparent background idea with text layering on top is prolly way beyond what a sensible rendering system in any TE can provide out of the box. So I will keep that as a funky idea for now.

Regarding "composition" from previous sixels I wrote a small test script: https://gist.github.com/jerch/443248dee1ff858d19be8b44487a5c7b

Note that the sixel introducer prolly needs some adjustments to work on a VT340. You can run the script with:

$> ./overprint.sh full | slim

full gives an idea, how slim ideally should look like. This works on xterm (after flipping h|l for sixel scrolling), and only there (tested several other TEs). slim is the actual overprinting mode, it does diff updates on the previous sixel. As far as I read the SIXEL spec (well DEC STD 070, if you want to call it a spec), this kind of diff overprinting should be possible. Thus I wondered, if thats actually the case for a VT340.

Edit: Why I am interested in such a mode, it promises some bandwidth savings for consecutive sixel painting, if changes are low between the frames.

[hackerb9]

Yup! You are correct. I see the whole emotional arc. Slim works better than full as there is no flickering.

[jerch]

Thx for the quick testing! Helps alot to get my TODO list straight. :smiley_cat:

[hackerb9]

By the way, it looks like XTerm 369 has flipped its interpretation of DECSDM so it now works the same as a VT340. This is good, but Sixel Display Mode should not actually be necessary for the bandwidth savings you are thinking about. In fact, it may be more useful to have it turned off as then you can use the standard text cursor positioning sequences to tell the terminal where to place the image.

[jerch]

Ah no worries, used it in the script only to get around needed text cursor re-adjustments. I am aware, that DECSDM has it own potential implementation pitfalls on TEs, so would prolly never use it in some serious app side code as of now.