inconsistent vertical cursor position after sixel display

[jdtournier]

What Operating System(s) are you seeing this problem on?

Linux Wayland

Which Wayland compositor or X11 Window manager(s) are you using?

Gnome Shell 46.4-1

WezTerm version

20240812-215703-30345b36

Did you try the latest nightly build to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version of the nightly that I tried

Describe the bug

After displaying a sixel image, the vertical cursor is often not in the expected position, and the image is subsequently (partially) overwritten by the prompt. This also happen when displaying multiple sixel images one after the other: each one overwrites the bottom (or the whole) of the next one.

To Reproduce

I'm using images of the Spleen bitmap fonts for demonstration purposes (8×16, 12×24, 16×32):

Using ImageMagick to display the image 3 times:

---

WezTerm:

8×16:

12×24:

16×32:

In the worst case, we only see one of the 3 repeats, and most of it is immediately overwritten by the prompt. For the 16×32 font, we can see each image overwrites the bottom of the previous one.

---

In contrast, we don't get that with e.g. mlTerm:

mlTerm:

8×16:

12×24:

16×32:

Configuration

no config

Expected Behavior

The vertical cursor should always be left at the expected position, after the last line of the sixel image.

Logs

Debug Overlay
wezterm version: 20240812-215703-30345b36 x86_64-unknown-linux-gnu
Window Environment: Wayland
Lua Version: Lua 5.4
OpenGL: AMD Radeon RX 590 Series (radeonsi, polaris10, LLVM 18.1.8, DRM 3.57, 6.10.8-arch1-1) 4.6 (Compatibility Profile) Mesa 24.2.2-arch1.1
Enter lua statements or expressions and hit Enter.
Press ESC or CTRL-D to exit

Anything else?

No response

[j4james]

Just FYI, it's expected that the cursor position should overlap the bottom of a sixel image (unless the application producing the image has explicitly added a newline), otherwise you wouldn't be able to render an image on the last row of the display without triggering a scroll. If you're comparing against mintty and mlterm, be aware that they're both setting the cursor position incorrectly. WezTerm's current behaviour isn't perfect in all edge cases, but it's at least better than those two terminals.

[jdtournier]

I must admit I'm no expert on sixel handling, and really just trying to piece it together from whatever information I can find online. So far, the only resource I've come across that explicitly mentions the cursor position following a sixel render is the minTTY wiki - but if you're suggesting that their handling is incorrect, then that doesn't necessarily help...

That said, they mention that the final cursor position can be:

1. next to the right bottom of the image
2. below the left bottom of the image (default)
3. at the line beginning below the image (like xterm)

As far as I can tell, option 1 is triggered in WezTerm when bottom_right (set here) is true, which for Sixel images occurs when sixel_scrolls_right is true. That can be triggered by issuing the ^[[?8452h control sequence, which does indeed result in the expected behaviour: the cursor ends up to the right of the image displayed, and the subsequent prompt shows up on that same line, to the right of the image. That's true for mlTerm, minTTY and WezTerm.

For the other two, I'm confused as to how that's handled. As far as I can tell, the default used to be option 3: a final newline was issued, and that caused a scroll (as mentioned in #3266 and fixed with a115fc0). As far as I can tell, this is what happens in mlTerm and minTTY as well. The fix I suggested in #6114 re-introduces that newline for Sixel images specifically.

But if that's not the correct behaviour, that's going to be a problem, since we're then going to have different outputs for the same program on different terminals... There aren't many sixel-capable terminals out there, and both minTTY & mlTerm feature prominently. The particular piece of code I'm working on is designed to work across a range of environments, primarily minTTY (via MSYS2), but also most likely iTerm2 on macOS, and I'd like to use WezTerm on my Linux setup as it's clearly a cut above the other options.

[jdtournier]

I'm also unsure about the difference between options 2 & 3 above. How is setting the cursor below the left bottom of the image different from setting it at the line below the image?

Also related: does the final sixel newline directive (i.e. a final - character before exiting sixel mode) constitute a newline in the sense that you intended here:

Just FYI, it's expected that the cursor position should overlap the bottom of a sixel image (unless the application producing the image has explicitly added a newline)

Finally, if we could replicate the way iterm images are handled, I'd be very happy with that. What I mean is that with iterm, the final cursor position is to the right of the image, and no newline is issued at that point - and there is no unexpected scrolling. But somehow the prompt appears on the next line anyway, and a scroll happens at that point. I think it would be great if we could leave the cursor position at the bottom left of the image (i.e. option 2 above), but not have the subsequent prompt immediately overwrite the bottom of the image. How is that achieved for iterm images, and is this something that can be done for sixel images as well? I couldn't figure out how to replicate that in my limited rummage through the WezTerm code - would it perhaps be sufficient to add:

          else {
                self.set_cursor_pos(
                    &Position::Relative(0),
                    &Position::Relative(y_padding_shift),
                );
          } 

in the else branch after this if statement? I'll test that when I get back to my desk...

[j4james]

For more information on the correct sixel cursor positioning, have a look at the test case here: https://github.com/hackerb9/vt340test/blob/main/j4james/cursor_position.sh

And you can see the resulting output from a real DEC VT340 terminal in the screen capture here: https://github.com/hackerb9/vt340test/blob/main/j4james/cursor_position.png

In short, the text cursor position is meant to be set to the same row as the top of the final sixel line. One of the neat properties of this algorithm is that it guarantees your cursor will be aligned with the bottom of the image if the image is a multiple of the text cell height (which is typically what you want in text UIs). From that position, a single LF will place the cursor exactly below the image.

1. next to the right bottom of the image
2. below the left bottom of the image (default)
3. at the line beginning below the image (like xterm)

These are just some of the different ways in which modern terminals have implemented sixel cursor positioning incorrectly.

1. I believe this was an invention of the RLogin terminal emulator, to work around the fact that their default cursor position (which was likely a variant of option 2) would trigger a scroll. So they added a mode which placed the cursor one row up and to the right. Some other terminal emulators have copied this, but not always in the same way. This is not a standard DEC mode.

2. This used to be one of the most common positions of modern terminal emulators (as seen in mintty and mlterm). But as I mentioned above, this triggers a scroll if you're trying to render an image on the last row of the page. This is not standard DEC behavior.

3. This is similar to option 2, except it adds a carriage return (i.e. with option 2 your starting column doesn't change; with option 3 the cursor is moved back to the leftmost column). I think this was once the default Xterm behavior, and Mintty supported it with mode 7730.

But note that Xterm doesn't use any of these algorithms by default anymore, and more closely matches the DEC behavior, although not exactly. I think it's similar to WezTerm's current algorithm.

Anyway, don't let me dissuade you from doing whatever you want here if you're doing so knowingly. I just wanted to make sure you were aware that your proposed changes were technically incorrect, and would potentially break applications that rely on terminals following the standard.

[jdtournier]

Thanks for taking the time to provide all the feedback and information, @j4james - I don't have anywhere near as much experience with terminals as you have, and it's really helpful to have all this background and context laid out.

I've had a go at various options, but I'm starting to realise that if wezterm is already behaving according to standard, there's not a lot of point in digging any further. I have no interest in introducing non-compliant behaviour if the issue is with the other terminals that I've been using... And as you said, xterm does indeed behave the same as wezterm.

Guess I'll just have to add a few more newlines in my own code... I'll close this issue and the associated PR, thanks for the guidance!