Removal of BlobLeases halts rendering of entire terminal

Link to original GH discussion

What Operating System(s) are you seeing this problem on?

macOS

Which Wayland compositor or X11 Window manager(s) are you using?

No response

WezTerm version

wezterm 20240128-202157-1e552d76

Did you try the latest nightly build to see if the issue is better (or worse!) than your current version?

Yes, and I updated the version box above to show the version of the nightly that I tried

Describe the bug

08:52:46.213  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)

To Reproduce

Nuclear approach: rm -rf $TMPDIR/wezterm-blob-lease-* Organic approach (on macOS): Leave WezTerm open for a few days with the laptop asleep, it'll eventually go blank

Configuration

I'm not sure whether each BlobLease is an individual frame or if they're due to changing hsb, but this is what I have in my config. Full config in the original discussion

config.background = {
  {
    source = {
      File = { path = assets .. "/blob_blue.gif", speed = 0.3 },
    },
    repeat_x = "Mirror",
    height = "100%",
    opacity = 0.10,
    hsb = {
      hue = 0.9,
      saturation = 0.9,
      brightness = 0.8,
    },
  },
}

Expected Behavior

Ideally, I'd expect the BlobLease(s) to be renewed when they are invalidated. Alternatively, having the exception caught and handled without blanking the entire terminal would be an acceptable solution.

Logs

08:52:43.531  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)
08:52:44.020  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)
08:52:44.341  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)
08:52:44.621  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)
08:52:46.131  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)
08:52:46.213  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)

Anything else?

No response

I have been plagued for months with this issue and always worked around it by force-quitting my Wezterm instance on system unlock in Mac and using zellij to ensure work is not lost. My situation sounds like yours:

I run nvim under zellij in wezterm. It works correctly and I eventually lock/sleep my Mac.
When I unlock my Mac after some amount of time, it fails to render the WezTerm window.
The window chrome renders with the inner pane invisible, though it still captures foreground focus.
Launching a new window of Wezterm also shows the window chrome without any pane rendering.
I have a terminal background image set in my wezterm config

Here are some screenshots, with personal data obscured:

My workstation upon unlock with wezterm in the foreground, and its pane invisible

A new wezterm window created after discovering the invisible pane

My logs indicate two separate, relevant lines:

The first is a blob lease error, probably because $TMPDIR was wiped:

09:20:48.789  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: render_backgrounds: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)

The second is the same error, but in populate_image_quad:

09:20:57.711  ERROR  wezterm_gui::termwindow::render::paint > paint_pass failed: paint_pane: error while calling with_lines_mut: render_screen_line: populate_image_quad: cached_image: frames.current_frame.lease.get_data: Io error in BlobLease: No such file or directory (os error 2): No such file or directory (os error 2)

What is weird, is that by repeatedly minimizing and maximizing the original window, I was able to get it to start rendering the original window's pane correctly. It is as if the focus transitions forced a cache flush and re-assignment.

A cursory glance in the source suggests issue in paint.rs where the LoadedBackgroundLayer appears to be loading a corrupted cached blob for an image background. However, I would expect this snippet to render a default background instead. Perhaps the window_background is empty and !loaded_any is true, in which case the fallback is failing to render a default when the BlobCache is corrupted.

I will be removing my terminal background image as a short-term measure to try and avoid the issue.

This should be resolved now in main.

It typically takes about an hour before commits are available as nightly builds for all platforms. Linux builds are the fastest to build and are often available within about 20 minutes. Windows and macOS builds take a bit longer.

Please take a few moments to try out the fix and let me know how that works out. You can find the nightly downloads for your system in the wezterm installation docs.

If you prefer to use packages provided by your distribution or package manager of choice and don't want to replace that with a nightly download, keep in mind that you can download portable packages (eg: a .dmg file on macOS, a .zip file on Windows and an .AppImage file on Linux) that can be run without permanently installing or replacing an existing package, and can then simply be deleted once you no longer need them.

If you are eager and can build from source then you may be able to try this out more quickly.

This should be resolved now in main.

Thank you! I agree that using an application-specific cache folder makes sense here. Tmp dir management will be OS-specific and for some time now Apple has not allowed this to be configurable anyways.

Edit: I will put back my background image and report back if the issue recurs on nightly. No news is good news :)

wez / wezterm