live resize/repaint "wobble"

[wez]

You can observe this in wezterm with a nightly build from here: https://github.com/wez/wezterm/releases/download/nightly/WezTerm-macos-nightly.zip

What happens is that when the window is resized, the size of the layer seems to be slightly out of sync with the new window size and that results in a series of brief moments where the content is scaled to the window size, making the text in the terminal appear to "wobble" or "ripple" until it catches up.

If I switch the application back to CGL, this wobble effect is eliminated.

Is there something that I should add to my application logic to force things to be more closely in sync?

FWIW, I also observe a similar effect with ANGLE and Direct3D on Windows, so this may not be strictly MetalANGLE specific.

[kakashidinho]

I think this is quite tricky to troubleshoot, since the resizing event is only happening for a short time. Do you really need an accurately rendering size during resizing? I think users wouldn't mind it since it only happens transiently.

May I know does your application redraw everything when a resize event occurs? How do you update the OpenGL viewport when the resize occurs? The issue could be caused by wrong viewport used at the time the resize just happened.

[wez]

When using CGL, this line of code in our resize handler is used to eventually trigger calling NSOpenGLContext::update:

https://github.com/wez/wezterm/blob/master/window/src/os/macos/window.rs#L1357

There doesn't appear to be an obvious place to trigger an equivalent update with MetalANGLE.

The resize handler instructs the other layers in the code about the new size, which will allocate new vertices if required and that indirectly causes view.setNeedsDisplay to be called to repaint the screen; that ultimately causes our draw_rect function to be called and it will update the viewport and render a frame.

[kakashidinho]

OK thanks, let me take a look, it could be a bug in MetalANGLE. The viewport and internal metal layer size might not be in sync at the moment the window resize just happened.

[wez]

Thanks; FWIW, I think it is reasonable request if the application needs to call something to give a hint about the resize (especially because CGL has this concept already today), so I'm happy to make the changes on the app side to trigger that if it can't be automagically deduced!

[wez]

Do you really need an accurately rendering size during resizing? I think users wouldn't mind it since it only happens transiently.

FWIW, I generally agree with this, but I think the audience for a terminal is generally biased towards critical eyes, so it has been reported as a minor issue already :-p

I would classify this as nice to fix, but not essential.

[kakashidinho]

Thanks; FWIW, I think it is reasonable request if the application needs to call something to give a hint about the resize (especially because CGL has this concept already today), so I'm happy to make the changes on the app side to trigger that if it can't be automagically deduced!

I cannot replicate CGL mechanism because ANGLE is EGL based, and EGL standard doesn't have any API to update backbuffer resize. At least doing so would require an EGL extension.

Though I'm not sure what's wrong yet. Could you try to call glClear() then glFlush() inside your did_resize. Calling glClear & glFlush could trigger a size change detection internally, then later your draw_rect by right should be able to proceed normally with resized internal metal layer. In short:

1. Inside did_resize:
   • CGL: NSOpenGLContext::update: ---> MetalANGLE: eglMakeCurrent (if you didn't do that already) + glClear + glFlush.

[wez]

I tried adding a clear + flush, but that causes a visible blank frame to render and makes the output flicker rather than render a slightly poorly scaled frame; so on the plus side, it doesn't wobble, but on the negative side it flickers :-p

I also tried triggering a draw immediately in the did_resize function, and a draw + flush + draw; the idea being that rendering the desired frame is better than rendering a blank frame, but both of those also result in a more flickery behavior than not doing any explicit flush/clear.

Not quite sure what to think about this right now; I'll ponder this some more.

[kakashidinho]

That’s strange. Clear + flush create a blank framebuffer but it should not display it. Only eglSwapBuffers will display the framebuffer on the screen. If you don’t do swap buffers by the time draw_rect is called, your draw_rect should overwrite the blank frame with a proper frame before displaying. I’m thinking there is an expected eglSwapBuffers somewhere and it could be the cause of both flickering and the original issue. I will try to troubleshoot locally also

[wez]

The glium library I'm using works hard to enforce consistency and is generating a call to swap buffers. I'll see if I can bypass that when I'm next sitting down with the mac.

[wez]

Not sure that I've formed a conclusion yet, but by adding in some debug prints in those callbacks I noticed that the ripple/wobble seems to coincide with cases like this where multiple calls to did_resize occur before a call to draw_rect:

 2020-10-30T16:27:55.597Z ERROR window::os::macos::window    > resize: backend.update
 2020-10-30T16:27:55.598Z ERROR window::os::macos::window    > call resize callback
 2020-10-30T16:27:55.603Z ERROR window::os::macos::window    > resize: backend.update
 2020-10-30T16:27:55.603Z ERROR window::os::macos::window    > call resize callback
 2020-10-30T16:27:55.608Z ERROR window::os::macos::window    > paint_opengl
 2020-10-30T16:27:55.611Z ERROR window::os::macos::window    > finish
 2020-10-30T16:27:55.611Z ERROR window::egl                  > swap_buffers

I did adjust the code to directly call glClear and glFlush; that happens on the lines that print backend.update. It didn't seem to impact things visually.

[kakashidinho]

Have just comeback to MetalANGLE recently after a break, I noticed you already canceled using MetalANGLE in your project :( It's a pity but I understand. Also I noticed your users are really sensitive to detailed issues like this (another example is 1 second delay in startup). Is the main reason people prefer your terminal is the fast performance compare to other terminals?

Nevertheless, your project is very cool.

[wez]

Yeah, MetalANGLE seemed to not be as snappy as CGL, and on Big Sur it looks like CGL is now effectively automatically translating to Metal under the covers (the vendor/implementation string reported by OpenGL has "Metal" in it now). I still have support for loading MetalANGLE, I just stopped shipping it by default because it seemed like a better default :-(

I believe that the main draw to wezterm is the combination of performance, features and cross platform support (for people that want consistency on multiple systems). I wouldn't claim that wezterm is top in any one of those categories on any given platform, but that it has good representation in all of them. For example: Alacritty is generally accepted as fastest and is also cross platform, but has fewer features by philosophy. On macOS, iTerm2 has more features with a more polished user experience; wezterm brings some of those features to other platforms.

While my personal goal isn't to make it the fastest thing ever, discerning CLI power users are very sensitive to latency!

[wez]

well, in a bit of a twist, it seems that CGL initialization can sometimes pause for ~10 seconds at a time on macOS 11.2 as we discovered in https://github.com/wez/wezterm/issues/452 so I'm going to see about getting an updated build of MetalANGLE going with an M1/universal binary.

[kakashidinho]

Hello, even though MetalANGLE is not used by wezterm by default anymore, but I cloned your latest version and tested with MetalANGLE, it seems the wobble issue no longer exist (or at least I see no difference between using MetaLANGLE and not using). Probably the past issue was caused by glium's wrong viewport detection (my suspicion). Could you check whether your latest version is already free of "wobble" when using MetalANGLE?