Next image corrupts previous rendered image. Images have missing scanlines

[aslpavel]

Describe the bug

There are two issues probably related

• If you have render image one after another, on the previously rendered images artifacts are appearing
• On lines that are supposed to be smooth discontinuities are visible
  • This might be due to nearest neighbor scaling, if it is the case then I think it is a bug, as I expect sixel images to pixel perfect and not scaled at all
  • This might also be due to showing texture on smaller than expected surface, which might produce nearest neighbor downscaling

Environment (please complete the following information):

• OS: macOS 10.15.7
• Version: wezterm 20200909-002054-4c9af461

To Reproduce

All of this is not specific to this image, in general if you render one image after another previous images are corrupted, and scaling is always weird

• Given source image
• If we render it twice one after another output will look like this
• Make note of this, looks like missing scan lines, or nearest neighbor downscaling scaling with smaller than expected height

Configuration

$ cat ~/.config/configs/wezterm.lua
local wezterm = require 'wezterm';

return {
   set_environment_variables = {
      COLORTERM="truecolor",
   },

   font_size = 9.5,
   font_antialias = "Subpixel",
   font_hinting = "Full",
   font = wezterm.font_with_fallback({
         "Iosevka",
         "Symbols Nerd Font",
   }),

   colors = {
      foreground = "#ebdbb2",
      background = "#282828",
      cursor_bg = "#ebdbb2",
      cursor_fg = "#ebdbb2",
      cursor_border = "#ebdbb2",
      scrollbar_thumb = "#504945",
      ansi = {"#282828", "#cc241d", "#98971a", "#d79921", "#458588", "#b16286", "#689d6a", "#ebdbb2"},
      brights = {"#7c6f64", "#fb4934", "#b8bb26", "#fabd2f", "#83a598", "#d3869b", "#8ec07c","#fbf1c7"},
      tab_bar = {
         background = "#1d2021",
         active_tab = {
            bg_color = "#076678",
            fg_color = "#ebdbb2",
         },
         inactive_tab = {
            bg_color = "#458588",
            fg_color = "#3c3836",
         },
         inactive_tab_hover = {
            bg_color = "#83a598",
            fg_color = "#3c3836",
         },
      }
   },

   ratelimit_output_bytes_per_second = 10000000,
   default_cursor_style = "SteadyBar",
   enable_scroll_bar = true,
}

Expected behavior

ITerm output as a reference

Session Recording

20201009110114.tgz

Additional examples where corruption is more visible

[aslpavel]

I've also noticed if you try to show img2sixel squirrel.sixel image around 60 times, terminal becomes unresponsive for a while, and all subsequent renders will draw blank space with the color of the default terminal background color.

[wez]

I haven't had an opportunity to dig into this, and I do want to see this improve, but in the meantime I thought you'd appreciate some background on how image support is implemented in wezterm:

At the parser layer:

• The decoded image is stored behind an Arc (shared pointer)
• Each terminal cell in the area displaying the image is set to an ImageCell that holds a ref on the image data as well as texture coordinates for the relevant portion of the image that should be rendered in that cell. That decision is based on the known cell size associated with the pty, which is an integer value.

The image data here is immutable so it is unlikely that it is being literally corrupted.

At the render layer:

• The image data is copied into a texture atlas
• Each cell has a pair of triangles that are updated with texture coordinates into the texture atlas

Something wonky with the texture coordinates at this stage might cause data to bleed in from neighboring data in the texture atlas; I think that might explain the fractals that appear at the bottom of your final image.

The "missing scan lines" are most likely an artifact of mapping the image to terminal cells some imperfect math/rounding issue in computing texture coordinates.

I suspect that you'd see improved results with a slightly different font size.

[aslpavel]

I think that might explain the fractals that appear at the bottom of your final image.

Fractals are from subsequent calls (that is I was rendering fractal animation after rendering squirrel), point here it is somehow ended up updating already rendered image, probably by using incorrect atlas coordinates

[aslpavel]

Another interesting point is querying terminal size with ioctl TIOCGWINSZ will report:

TerminalSize { cells: Size { height: 30, width: 229 }, pixels: Size { height: 1018, width: 2977 } }

If you convert it to cell size it will result in fractional value: height=33.933333... width=13

[wez]

In master, with the revised texture atlas allocation algorithm, the "corruption" case is very easy to trigger:

cd libsixel
cat images/snake.six
wezterm imgcat images/egret.jpg

The top few pixels of egret.jpg now appear in the bottom few pixels (after a padding break) of the snake graphic. This is due to the bottom few pixels of snake being smaller than the cell height. This is something wonky with the texture coordinates selected for the bottom row.

(you can also reproduce by cat'ing snake twice)

[wez]

I believe that both the scaling and the bleed-through are resolved in master; at least it appears to work for me. I'd appreciate it if you could give it a try when you have a chance!

[aslpavel]

Thank you for the fix, but I was not able to check if it is working. I'm using MacOS and apparently 91ce0b77eaab95b94432b5a0379d64129e07f3e7 breaks it for me. It is panics with 'No OpenGL' previous revision works fine. It produces the following error:

2020-10-24T23:15:43.291Z ERROR wezterm::gui::termwindow > scale changed so resize to RowsAndCols { rows: 24, cols: 80 } Dimensions { pixel_width: 1134, pixel_height: 900, dpi: 192 }
 2020-10-24T23:15:43.340Z WARN  window::os::macos::window::opengl > EGL init failed: with_egl_lib failed: /Users/aslpavel/wezterm/target/release/libEGL.dylib: dlopen(/Users/aslpavel/wezterm/target/release/libEGL.dylib, 2): image not found, libEGL.dylib: dlopen(libEGL.dylib, 2): image not found, /Users/aslpavel/wezterm/target/release/libEGL.dylib: dlopen(/Users/aslpavel/wezterm/target/release/libEGL.dylib, 2): image not found, libEGL.dylib: dlopen(libEGL.dylib, 2): image not found, falling back to CGL
 2020-10-24T23:15:44.899Z INFO  wezterm::gui::renderstate         > compiling a prog with version 330
 2020-10-24T23:15:44.899Z INFO  wezterm::gui::renderstate         > compiling a prog with version 300 es
 2020-10-24T23:15:44.899Z ERROR wezterm::gui::termwindow          > failed to create OpenGLRenderState: Failed to compile shaders: Compilation error in vertex shader: ERROR: 0:4: 'attribute' : syntax error: syntax error

Compilation error in vertex shader: ERROR: 0:1: '' :  version '300' is not supported
ERROR: 0:1: '' : syntax error: #version
ERROR: 0:2: '' :  #version required and missing.
ERROR: 0:4: 'attribute' : syntax error: syntax error

thread 'main' panicked at 'No OpenGL', wezterm/src/gui/termwindow.rs:720:41
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at 'cannot access a Thread Local Storage value during or after destruction: AccessError', /Users/aslpavel/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/std/src/thread/local.rs:241:26
 2020-10-24T23:15:45.002Z ERROR mux                               > read_pty EOF: pane_id 0
fatal runtime error: failed to initiate panic, error 5
Abort trap: 6

[wez]

doh, thanks for reporting that! I'll revert that portion of the change :-/

[aslpavel]

It looks like fix works, I no longer see any corruptions. Thank you for the fix.

[wez]

Thanks for confirming!

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.