search

contour-terminal / contour

Modern C++ Terminal Emulator
http://contour-terminal.org/
Apache License 2.0
2.39k stars 101 forks source link

sixel painting to not destroy existing contents #450

Closed christianparpart closed 2 years ago

christianparpart commented 2 years ago

you'll then see that Contour draws the image on a black background, annihilating the glyphs underneath, unlike other sixel implementations which interpret unspecified pixels as transparent =]. fixing that would be the thing that earned you the most high marks in my book =].

Challenge excepted. 💣 I see what I can do to get it into the next bugfix cycle (release 0.2.1), as that can be seen as a fix too, because that's how xterm is rendering it, right?

j4james commented 2 years ago

Copying my comment from the notcurses discussion, in case you don't see it there:

Note that transparency is dependent on the Background Select parameter (P2) of the sixel escape sequence. The background should only be transparent if P2 is set to 1. Xterm doesn't handle this correctly, so please don't copy their behaviour exactly.

christianparpart commented 2 years ago

image

Wow. I expected it to take longer. :partying_face:

p.s.: @dankamongmen please give me 3x2+ and ucurl+. I git pull daily to see your updates :-)

christianparpart commented 2 years ago

OTOH: just thinking about it.... the PR as of right now contains a ninja-commit that'll be moved into another PR. But I'm not 100% sure I'm doing it right. I do respect P2 (second parameter I suppose you mean) such that if it is equal to 1 then bg pixels will be 0/0/0/0 (RGB black 0% opacity), which I think is sufficient unless I did misunderstand something. p.s.: didn't look into xterm source code :)

j4james commented 2 years ago

I've actually been having second thoughts recently regarding my understanding of the P2 parameter. The VT340 documentation implies that you set the pixel to the background color when P2 is 0 or 2, and leave it unchanged when P2 is 1, but I don't think that's strictly correct. My theory now is that P2 simply controls whether the background is filled (which is explains why it's called Background Select), and 0 pixels should never be output.

If the background has already been filled, and you don't output a pixel, that's still the same as outputting the background color (which matches the described behavior in the documentation). But if your image extends outside the background boundaries, then some 0 pixels may end up being transparent, even when P2 is 0 or 2. And if you've set raster attributes, you're always going to be filling the background to some extent, so P2 would theoretically have no effect then.

Anyway, it's just a theory at the moment, but it lines up with some of the results I've seen in our testing on the VT340. When I have more time I'll try and confirm it one way or the other.

christianparpart commented 2 years ago

Hey. Many thanks for your thoughts though. I am curious for further research results to come.

Long term goal is still too continue working on the Grid Image Protocol. But Sixel i know think it's still good too have and you have it confirmation you some extend - as long as we all can agree on what "confirm" actually kids.

dankamongmen commented 2 years ago

But if your image extends outside the background boundaries, then some 0 pixels may end up being transparent, even when P2 is 0 or 2. And if you've set raster attributes, you're always going to be filling the background to some extent, so P2 would theoretically have no effect then.

could you please expand on this a bit? in particular, what does it mean for an "image to extend outside the background boundaries"? oh does this mean if your image exceeds the geometry that you included in Ph and Pv? then yeah, my reading matches your interpretation, i think.

j4james commented 2 years ago

Note that I was wrong about P2 not having any effect when raster attributes are set. It looks more like it's the other way around, i.e. the dimensions specified in the raster attributes have no effect when P2 is set to 1.

For example, consider an image like this over a background of Es: echo -e '\ePq"5;1;120;60#1!60T-!60T-!60T-!60T\e\\\n'

image

The image is 120px high while the raster attribute height is given as 60px, so the half that extends below that point will have the underlying text showing through, despite the fact that P2 is not set to 1.

But if you do set P2 to 1, I suspect the background will not be filled at all, so all the zero pixels in the image will be treated as transparent, and the dimensions specified by the raster attributes are irrelevant. echo -e '\eP;1q"5;1;120;60#1!60T-!60T-!60T-!60T\e\\\n'

image

christianparpart commented 2 years ago

I have to sadly admit I am ignoring the aspect ratio (if provided) currently. I take the pixels as is. I am absolutely fine in fixing that. But until now it seemed (!) sufficient for modern apps as that's never set anyways.

j4james commented 2 years ago

I have to sadly admit I am ignoring the aspect ratio

Not many TEs handle aspect ratio correctly, so if you don't intend to support legacy VT340 and VT240 applications, it's probably not a big deal. Occasionally you may come across old sixel images from the VT240 that depend on AR, because it only supported 2:1, but I don't think it's that common. I just find it useful for test cases, because it's so convenient when you just need to output simple block structures, so I use it all the time.

dankamongmen commented 2 years ago

Note that I was wrong about P2 not having any effect when raster attributes are set. It looks more like it's the other way around, i.e. the dimensions specified in the raster attributes have no effect when P2 is set to 1

this matches my intuition (save that the terminal might use them to optimize something).

dankamongmen commented 2 years ago

I have to sadly admit I am ignoring the aspect ratio (if provided) currently. I take the pixels as is. I am absolutely fine in fixing that. But until now it seemed (!) sufficient for modern apps as that's never set anyways.

fwiw, i never use them, since it doesn't make any sense to me how aspect ratio could be enforced on a pixel (and images don't always include information on their "pixel aspect ratio").

j4james commented 2 years ago

it doesn't make any sense to me how aspect ratio could be enforced on a pixel

Think of it as similar to line doubling. If the AR is 2:1, every scanline in the image is output twice, if it's 3:1 they'll be output three times, etc. Aspect ratios are only ever integer values.

images don't always include information on their "pixel aspect ratio"

If it's not specified, I believe the default is meant to be 2:1, for backwards compatibility with the VT240 (possibly also the VT125).

dankamongmen commented 2 years ago

Think of it as similar to line doubling. If the AR is 2:1, every scanline in the image is output twice, if it's 3:1 they'll be output three times, etc. Aspect ratios are only ever integer values.

ahhh ok so the result isn't necessarily what you specify; this is strictly a blowup. got it. still kinda neat!

If it's not specified, I believe the default is meant to be 2:1, for backwards compatibility with the VT240 (possibly also the VT125).

i meant the source material, i.e. pngs and such.