gfx-rs / wgpu

A cross-platform, safe, pure-Rust graphics API.
https://wgpu.rs
Apache License 2.0
12.21k stars 896 forks source link

Surface::get_current_texture does not block when the window is fully invisible (macOS) #4779

Open ComfyFluffy opened 10 months ago

ComfyFluffy commented 10 months ago

Description I am currently following a WGPU Tutorial and have implemented Hello Triangle with vertex buffers.

image

When I click the green button to expand the window into fullscreen, the process will suddenly take up to 22GB of memory:

image

After a few seconds, the memory usage goes down:

image image

Repro steps I have uploaded the code onto GitHub.

git clone https://github.com/ComfyFluffy/wgpu-tutorial.git
cd wgpu-tutorial
RUST_LOG=info MTL_HUD_ENABLED=1 cargo run

Then expand the app to fullscreen.

If expanding to fullscreen does not cause memory peak, switching to other desktop then switch back to the app can also cause it.

Resizing on windowed mode does not trigger the behavior.

Resizing on Windows 11 does not trigger the behavior.

Expected vs observed behavior

Expected: The memory usage should remain largely the same during resizing. Observed: The memory usage goes up to 22 GB.

Extra materials Info log logged by env_logger. Memory peak happened during resizing.

[2023-11-26T06:43:52Z INFO  wgpu_core::instance] Adapter Metal AdapterInfo { name: "Apple M1 Pro", vendor: 0, device: 0, device_type: IntegratedGpu, driver: "", driver_info: "", backend: Metal }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: Fifo, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:52Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:52Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1200, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:53Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:53Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1200, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 1600, height: 1256, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 1600, height: 1256, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] configuring surface with SurfaceConfiguration { usage: TextureUsages(RENDER_ATTACHMENT), format: Bgra8UnormSrgb, width: 5120, height: 2880, present_mode: AutoVsync, alpha_mode: Opaque, view_formats: [] }
[2023-11-26T06:43:56Z INFO  wgpu_core::device::global] Automatically choosing presentation mode by rule AutoVsync. Chose Fifo
[2023-11-26T06:43:56Z INFO  wgpu_hal::metal::surface] build swapchain SurfaceConfiguration { swap_chain_size: 3, present_mode: Fifo, composite_alpha_mode: Opaque, format: Bgra8UnormSrgb, extent: Extent3d { width: 5120, height: 2880, depth_or_array_layers: 1 }, usage: TextureUses(COLOR_TARGET), view_formats: [] }

Platform Macbook Pro 14 (2021), M1 Pro with macOS Sonoma, 16 GB Memory wgpu: 0.18.0

ComfyFluffy commented 10 months ago

Setting to fullscreen with window.set_fullscreen(Some(winit::window::Fullscreen::Borderless(None))) (winit window) on macOS will also cause the issue, but not on Windows 11.

The issue does not present when using Vulkan with MoltenVK.

cwfitzgerald commented 10 months ago

Try again now that https://github.com/gfx-rs/wgpu/pull/4781 has landed

cwfitzgerald commented 10 months ago

This doesn't reproduce for me basically identical setup on trunk. Going to close, if you tell me it's still happeneing I'll reopen.

ComfyFluffy commented 10 months ago

This still happens to me after updating wgpu (2964eed). Using macOS 14.1.1.

Sometimes resizing to fullscreen only will not trigger the issue. However, the issue will present swapping to other desktop using trackpad and then swapping back to the fullscreen app.

I have updated my git repo of the demo.

ComfyFluffy commented 10 months ago
image image

With further investigation using Instrument I found that VM: IOSurface has 9 GB in this run, which traces into CA::SurfaceUtil::CAIOSurfaceCreate(unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned long long, CA::SurfaceUtil::SurfaceAlignment, __CFString const*).

The stack trace in a single call:

image
ComfyFluffy commented 10 months ago

With further investigation I located the issue to the execution time of surface.get_current_texture().

In the event loop, I use the following structure to render:

match event {
    Event::WindowEvent { event, .. } => match event {
        WindowEvent::RedrawRequested => {
            match state.render() {
                Ok(_) => {}
                Err(wgpu::SurfaceError::Lost) => {
                    warn!("Lost surface");
                    state.resize(state.size)
                }
                Err(wgpu::SurfaceError::OutOfMemory) => panic!("Out of memory"),
                Err(e) => eprintln!("{:?}", e),
            }
            println!(
                "Rendered in {}ms",
                render_start.elapsed().as_micros() as f32 / 1000.0
            );
        }
        _ => {}
    },
    Event::AboutToWait => {
        state.window().request_redraw();
    }
    _ => {}
}

Generally, state.render() takes about 8ms to finish (on 120hz monitor). Inside render() I use surface.get_current_texture() to get output texture. Normally it takes about 8ms to get the texture before rendering:

image

However, when putting the window into another desktop, or setting fullscreen mode then switch to other desktops, the window becomes fully invisible. This cause the time of calling surface.get_current_texture() changes to only a few microseconds:

image

Since the render has finished, Event::AboutToWait triggers, causing it to render again. This results too much frames being draw in a short period of time. Each get_current_texture() call allocates about 22M of memory, overwhelming the system memory.

Nnubes256 commented 5 months ago

For those wondering how to mitigate this issue if affected by it while a proper fix lands, this comment I made for the previous big memory leak may help: https://github.com/sotrh/learn-wgpu/issues/207#issuecomment-1235400560.

The key thing of these two mitigations is avoiding drawing or limiting the number of frames drawn when the window is occluded, which helped with the previous issue; in this case, it might help in this case if you can detect whether the window is unfocused (thus, in particular, the CVDisplayLink hack I made may fix the issue entirely). Bevy mitigated this, for example, by limiting the rate of frames rendered when the app was in the background, see https://github.com/bevyengine/bevy/pull/7611.