[mtl] hangs on background/foregrounding in `[CAMetalLayer nextDrawable]`

[mtak-]

Short info header:

• GFX version: ll - rev: "bd7f058efe78d7626a1cc6fbc0c1d3702fb6d2e7"
• OS: macos (probly ios too)
• GPU: Radeon Pro 455 2048 MB

While an OSX app is backgrounded, attempting to call into -[CAMetalLayer nextDrawable] causes a hang (~1sec presumably a timeout).

The incorrect synchronization in the quad example masks the issue a bit. The hang is more obvious in this gfx-hal-tutorial example.

command_queues[0].submit(submission, Some(in_flight_fence)); // hangs sometimes

You would think that disabling drawing after the Focused(false) event is sent would fix the issue, but, sometimes the backgrounding event races with command buffer submission. After the hang occurs, then the Focused(false) event gets received - sometimes.

Apple really really wants you to draw only when they tell you to draw. There are a few options for being notified that OSX is expecting a new frame.

• CVDisplayLink/CADisplayLink api - cube example from moltenvk
  • this has the downside of running in another thread on OSX
• NSViewDelegate::draw/NSView::drawRect / UIViewDelegate::draw / UIView::drawRect
  • glfw example
  • metalkit tutorial
• NSView::updateLayer - glfw

I hacked up a CVDisplayLink impl to test and verified that the render thread was no longer hanging.

I believe the correct place for a fix should be in winit, but I don't see how it would work the current API.

Posting this here in case anybody has some opinions or ideas.

[kvark]

Thank you for such a detailed issue! I believe it's still our responsibility to avoid requesting the next drawable if we know that the metal layer is hidden. We should be able to check for it at the surface level, perhaps returning SurfaceLost error.

[floooh]

FYI I'm currently investigating the same issue in my experimental multi-window branch in sokol_app.h. When one of the windows is becoming obscured, the main thread stops for somewhere between half a second and a full second in [MTKView currentRenderPassDescriptor] (which presumably is calling [CAMetalLayer nextDrawable], which causes the freeze).

I guess the problem also exists with a single window, but it's not noticeable when rendering stops on an obscured window.

One strange thing I'm seeing in Instruments is, that the pause seems to be related to "vsync requests" stopping:

Those "vsync requests" happen in 16.6 ms intervals, until a window becomes obscured, than the vsync requests seems to "dry out" for a while, until they start happening at regular intervals again after usually around a second. Since no new drawables are presented, new drawables get stuck waiting to be presented, causing the whole thread to stop in [nextDrawable].

I don't know if I'm interpreting this correctly though. Googling for "vsync requests" doesn't yield much useful information, and the Instruments documentation also doesn't help much :/

I haven't found a solution so far, it doesn't seem to help to skip drawing when a window is obscured (via NSWindow.occlusionState), it seems it's already too late when this triggers to "occluded" :/

[floooh]

Some new info before I file this problem under "known issue" and move on:

The short freeze even happens in the official Xcode "Game" example (which uses MTKView. MTKView isn't the problem, but the nextDrawable method of CAMetalLayer.

To reproduce: add the following lines around the [MTKView currentRenderPassDescriptor] (which calls nextDrawable under the hood):

...then run the sample, and occlude its window with another window. Each time the window is fully occluded, the logging output will stop with the "before..." line for around a second and then continue (meaning that nextDrawable blocks for around a second each time the window becomes occluded).

TBH I'm not sure now if this isn't a bug in macOS that should be fixed by Apple (I'm on the 11.4 Beta (20F5046g)).

[lunacookies]

I’ve been banging my head against a wall the past few weeks trying on and off to solve a similar issue (nextDrawable hanging for one second after entering or exiting full screen), and I finally found the solution: stop your own render loop while the window is in transition. Instead, let the OS tell you when to draw using needsDisplayOnBoundsChange so that the window remains updated during e.g. custom full screen animations.

In my window delegate I have something like the following:

- (void)windowWillEnterFullScreen:(NSNotification *)notification
{
    [view stopDisplayLink];
}

- (void)windowDidEnterFullScreen:(NSNotification *)notification
{
    [view startDisplayLink];
}

- (void)windowWillExitFullScreen:(NSNotification *)notification
{
    [view stopDisplayLink];
}

- (void)windowDidExitFullScreen:(NSNotification *)notification
{
    [view startDisplayLink];
}

which corresponds to the following in my view:

- (void)startDisplayLink
{
    CVDisplayLinkStart(displayLink);
}

- (void)stopDisplayLink
{
    CVDisplayLinkStop(displayLink);
}

FWIW I haven’t been able to reproduce memory leaks / hangs after re-focusing the window following a period of occlusion.