search

bevyengine / bevy

A refreshingly simple data-driven game engine built in Rust
https://bevyengine.org
Apache License 2.0
36.24k stars 3.57k forks source link

Bevy application doesn't terminate upon window close when using AMDVLK Vulkan driver #10260

Closed robojeb closed 6 months ago

robojeb commented 1 year ago

Bevy version

Mainline Bevy

git+https://github.com/bevyengine/bevy.git#66f72dd25bb9e3f3d035f2f14dcbcd25674f968c

[Optional] Relevant system information

What you did

Ran Bevy with the AMDVLK Vulkan driver, and closed the game window.

What went wrong

The game window closes, but the application continues to run until ctrl+c

2023-10-25T18:25:58.262755Z  INFO bevy_winit::system: Creating new window "App" (0v0)
2023-10-25T18:25:58.263005Z  INFO winit::platform_impl::platform::x11::window: Guessed window scale factor: 1    
2023-10-25T18:25:58.348593Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 7900 XT", vendor: 4098, device: 29772, device_type: DiscreteGpu, driver: "AMD open-source driver", driver_info: "2023.Q3.3 (LLPC)", backend: Vulkan }
2023-10-25T18:26:01.907275Z  INFO bevy_window::system: No windows are open, exiting
2023-10-25T18:26:01.907772Z  INFO bevy_winit::system: Closing window 0v0
<ctrl+c required here>

Additional information

Forcing the vulkan-radeon driver using environment variables makes the issue go away. 

> AMD_VULKAN_ICD=RADV cargo run
2023-10-25T18:24:43.999447Z  INFO bevy_winit::system: Creating new window "App" (0v0)
2023-10-25T18:24:43.999663Z  INFO winit::platform_impl::platform::x11::window: Guessed window scale factor: 1    
2023-10-25T18:24:44.080668Z  INFO bevy_render::renderer: AdapterInfo { name: "AMD Radeon RX 7900 XT (RADV GFX1100)", vendor: 4098, device: 29772, device_type: DiscreteGpu, driver: "radv", driver_info: "Mesa 23.2.1-arch1.2", backend: Vulkan }
2023-10-25T18:24:46.288350Z  INFO bevy_window::system: No windows are open, exiting
2023-10-25T18:24:46.288592Z  INFO bevy_winit::system: Closing window 0v0

So this issue appears to only be related to the AMDVLK driver option.

alice-i-cecile commented 1 year ago

Very interesting: I've seen this issue reported for Linux before, but this is by far the best investigation I've seen. Not entirely sure what should be done to fix this though 🤔

robojeb commented 1 year ago

The issue does reproduce with multi_threaded removed from the project features.

I grabbed a single threaded stack trace from gdb: 

#0  0x00007fcd5f7564ae in ?? () from /usr/lib/libc.so.6
#1  0x00007fcd5f761980 in ?? () from /usr/lib/libc.so.6
#2  0x00007fcd5acbe750 in ?? () from /usr/lib/amdvlk64.so
#3  0x00007fcd5abb201e in ?? () from /usr/lib/amdvlk64.so
#4  0x00007fcd5a9856ab in ?? () from /usr/lib/amdvlk64.so
#5  0x00007fcd4de797fe in DispatchDestroySwapchainKHR (device=device@entry=0x564b0f065d60, swapchain=<optimized out>, swapchain@entry=0xfef35a00000000a0, 
    pAllocator=pAllocator@entry=0x0)
    at /usr/src/debug/vulkan-validation-layers/Vulkan-ValidationLayers-sdk-1.3.261.1/layers/layer_chassis_dispatch_manual.cpp:393
#6  0x00007fcd4dd840ef in vulkan_layer_chassis::DestroySwapchainKHR (device=0x564b0f065d60, swapchain=0xfef35a00000000a0, pAllocator=0x0)
    at /usr/src/debug/vulkan-validation-layers/Vulkan-ValidationLayers-sdk-1.3.261.1/layers/vulkan/generated/chassis.cpp:5689
#7  0x00007fcd61b5eeb7 in ash::extensions::khr::swapchain::Swapchain::destroy_swapchain ()
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/ash-0.37.3+1.3.251/src/extensions/khr/swapchain.rs:70
#8  wgpu_hal::vulkan::instance::{impl#5}::unconfigure (self=<optimized out>, device=<optimized out>) at src/vulkan/instance.rs:808
#9  0x00007fcd619c2c26 in wgpu_core::hub::Hub<wgpu_hal::vulkan::Api, wgpu_core::identity::IdentityManagerFactory>::surface_unconfigure<wgpu_hal::vulkan::Api, wgpu_core::identity::IdentityManagerFactory> (self=<optimized out>, device_id=..., surface=0x7fff77597ec0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/hub.rs:659
#10 0x00007fcd61a48803 in wgpu_core::instance::{impl#6}::surface_drop::unconfigure<wgpu_core::identity::IdentityManagerFactory, wgpu_hal::vulkan::Api> (
    global=<optimized out>, surface=<optimized out>, present=<optimized out>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/instance.rs:653
#11 wgpu_core::global::Global<wgpu_core::identity::IdentityManagerFactory>::surface_drop<wgpu_core::identity::IdentityManagerFactory> (self=0x564b0f0045f0, 
    id=...) at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-core-0.17.1/src/instance.rs:659
#12 0x00007fcd614e9db1 in core::ptr::drop_in_place<wgpu::Surface> () at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#13 0x00007fcd61504062 in core::ptr::drop_in_place<bevy_render::view::window::SurfaceData> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#14 core::ptr::drop_in_place<core::option::Option<bevy_render::view::window::SurfaceData>> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ptr/mod.rs:498
#15 bevy_render::view::window::WindowSurfaces::remove (self=0x564b0f171570, window=<optimized out>) at src/view/window/mod.rs:209
#16 0x00007fcd614d579b in bevy_render::view::window::extract_windows (extracted_windows=..., screenshot_manager=..., closed=..., windows=..., removed=..., 
    window_surfaces=...) at src/view/window/mod.rs:173
#17 core::ops::function::FnMut::call_mut<fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), (bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:166
#18 core::ops::function::impls::{impl#3}::call_mut<(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (
    self=<optimized out>, args=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:294
#19 bevy_ecs::system::function_system::{impl#21}::run::call_inner<(), bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>, &mut fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (f=<optimized out>, F0=..., F1=..., F2=..., F3=..., F4=..., F5=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:641
#20 bevy_ecs::system::function_system::{impl#21}::run<(), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::--Type <RET> for more, q to quit, c to continue without paging-- 
RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>> (
    self=<optimized out>, param_value=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:644
#21 bevy_ecs::system::function_system::{impl#6}::run_unsafe<fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>), fn(bevy_ecs::change_detection::ResMut<bevy_render::view::window::ExtractedWindows>, bevy_render::extract_param::Extract<bevy_ecs::change_detection::Res<bevy_render::view::window::screenshot::ScreenshotManager>>, bevy_render::extract_param::Extract<bevy_ecs::event::EventReader<bevy_window::event::WindowClosed>>, bevy_render::extract_param::Extract<bevy_ecs::system::query::Query<(bevy_ecs::entity::Entity, &bevy_window::window::Window, &bevy_window::raw_handle::RawHandleWrapper, core::option::Option<&bevy_window::window::PrimaryWindow>), ()>>, bevy_render::extract_param::Extract<bevy_ecs::removal_detection::RemovedComponents<bevy_window::raw_handle::RawHandleWrapper>>, bevy_ecs::change_detection::ResMut<bevy_render::view::window::WindowSurfaces>)> (
    self=0x564b0f3369c0, input=<optimized out>, world=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/system/function_system.rs:484
#22 0x00007fcd6230cfe0 in bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure#0} () at src/schedule/executor/single_threaded.rs:92
#23 core::ops::function::FnOnce::call_once<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}, ()> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#24 core::panic::unwind_safe::{impl#23}::call_once<(), bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}> (self=...)
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/panic/unwind_safe.rs:271
#25 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>, ()> (data=<optimized out>) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panicking.rs:504
#26 std::panicking::try<(), core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>> (
    f=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panicking.rs:468
#27 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<bevy_ecs::schedule::executor::single_threaded::{impl#0}::run::{closure_env#0}>, ()> (
    f=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/std/src/panic.rs:142
#28 bevy_ecs::schedule::executor::single_threaded::{impl#0}::run (self=0x564b0f04d910, schedule=0x7fff775991a0, world=0x564b0f222350)
    at src/schedule/executor/single_threaded.rs:91
#29 0x00007fcd6172766e in bevy_ecs::world::{impl#3}::run_schedule::{closure#0}<bevy_render::ExtractSchedule> (world=0x564b0f222350, sched=0x7fff77599180)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2088
#30 bevy_ecs::world::World::try_schedule_scope<(), bevy_render::ExtractSchedule, bevy_ecs::world::{impl#3}::run_schedule::{closure_env#0}<bevy_render::ExtractSchedule>> (self=0x564b0f222350, label=..., f=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2007
#31 bevy_ecs::world::World::schedule_scope<(), bevy_render::ExtractSchedule, bevy_ecs::world::{impl#3}::run_schedule::{closure_env#0}<bevy_render::ExtractSchedule>> (self=0x564b0f222350, label=..., f=...)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2059
#32 0x00007fcd615724d3 in bevy_ecs::world::World::run_schedule<bevy_render::ExtractSchedule> (self=<optimized out>)
    at /home/jeb/.local/share/cargo/git/checkouts/bevy-f7ffde730c324c74/66f72dd/crates/bevy_ecs/src/world/mod.rs:2088
#33 bevy_render::extract (main_world=0x7fff7759b718, render_app=0x564b0f222350) at src/lib.rs:396
#34 0x00007fcd622a443b in alloc::boxed::{impl#49}::call<(&mut bevy_ecs::world::World, &mut bevy_app::app::App), (dyn core::ops::function::Fn<(&mut bevy_ecs::world::World, &mut bevy_app::app::App), Output=()> + core::marker::Send), alloc::alloc::Global> (self=0x564b0f222340, args=...)
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/alloc/src/boxed.rs:2021
#35 bevy_app::app::SubApp::extract (self=0x564b0f222340, main_world=0x7fff7759b718) at src/app.rs:166
#36 bevy_app::app::App::update (self=0x7fff7759b718) at src/app.rs:264
#37 0x00007fcd60592720 in bevy_winit::winit_runner::{closure#0} (event=..., event_loop=0x564b0d97cea0, control_flow=0x7fff7759a740) at src/lib.rs:818
#38 0x00007fcd605c887e in winit::platform_impl::platform::sticky_exit_callback<(), bevy_winit::winit_runner::{closure_env#0}> (evt=..., target=0x0, 
    control_flow=0x7fff7759a740, callback=0x7fff7759afd0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/mod.rs:884
#39 winit::platform_impl::platform::x11::{impl#2}::run_return::single_iteration<(), bevy_winit::winit_runner::{closure_env#0}> (this=0x7fff7759ab80, 
    control_flow=0x7fff7759a740, cause=0x7fff7759a7f0, callback=0x7fff7759afd0)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:358
#40 0x00007fcd605c8ea8 in winit::platform_impl::platform::x11::EventLoop<()>::run_return<(), bevy_winit::winit_runner::{closure_env#0}> (
    self=0x7fff7759ab80, callback=...)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:483
#41 0x00007fcd605ca98d in winit::platform_impl::platform::x11::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (self=..., 
    callback=<error reading variable: Cannot access memory at address 0x1a1>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/x11/mod.rs:498
#42 0x00007fcd60584368 in winit::platform_impl::platform::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (
    self=<error reading variable: Cannot access memory at address 0x0>, callback=<error reading variable: Cannot access memory at address 0x18>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/platform_impl/linux/mod.rs:792
#43 winit::event_loop::EventLoop<()>::run<(), bevy_winit::winit_runner::{closure_env#0}> (
    event_handler=<error reading variable: Cannot access memory at address 0x18>)
    at /home/jeb/.local/share/cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.28.7/src/event_loop.rs:305
#44 bevy_winit::run<bevy_winit::winit_runner::{closure_env#0}, ()> (event_handler=<error reading variable: Cannot access memory at address 0x18>)
    at src/lib.rs:235
#45 bevy_winit::winit_runner (app=...) at src/lib.rs:902
#46 0x00007fcd60575902 in core::ops::function::FnOnce::call_once<fn(bevy_app::app::App), (bevy_app::app::App)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#47 core::ops::function::FnOnce::call_once<fn(bevy_app::app::App), (bevy_app::app::App)> ()
    at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/core/src/ops/function.rs:250
#48 0x00007fcd622a45f9 in alloc::boxed::{impl#47}::call_once<(bevy_app::app::App), (dyn core::ops::function::FnOnce<(bevy_app::app::App), Output=()> + core::marker::Send), alloc::alloc::Global> (self=..., args=...) at /rustc/49691b1f70d71dd7b8349c332b7f277ee527bf08/library/alloc/src/boxed.rs:2007
--Type <RET> for more, q to quit, c to continue without paging--
#49 bevy_app::app::App::run (self=<optimized out>) at src/app.rs:315
#50 0x0000564b0be29272 in general_logistics::main () at src/bin/general_logistics.rs:5

Its getting stuck deep in the AMDVLK code, so it might be their issue? Unless wgpu is doing something "non conforming" when destroying the swapchain, that other drivers handle better.

For now I will probably just go back to using radeon-vulkan, I only installed AMDVLK for some raytracing tests and I don't need to keep it installed.

robojeb commented 1 year ago

I created a minimal example that can control the issue by changing the window PresentMode

use bevy::{prelude::*, window::PresentMode};

fn main() {
    App::new()
        .add_plugins(DefaultPlugins.set(WindowPlugin {
            primary_window: Some(Window {
                present_mode: PresentMode::AutoNoVsync,  /* Works */
                ..default()
            }),
            ..default()
        }))
        .run();
}

The issue only reproduces with PresentMode::AutoVsync and PresentMode::Fifo (which I think is the default in AutoVsync mode). The issue also doesn't appear to reproduce if the WinitSettings::desktop_app() resource is provided.

Interestingly, the issue does not reproduce running wgpu boids or cube examples, even though they should be using PresentMode::Fifo as well. I checked on both wgpu 0.17.1 and trunk.

I haven't been able to narrow down what Bevy is doing that causes FIFO mode to apparently deadlock when the swap-chain is destroyed.

robojeb commented 1 year ago

@alice-i-cecile I think I have narrowed the issue.

In PresentMode::Fifo when the winit::window::Window structure is dropped before wgpu::Surface, then the AMDVLK driver will hang while trying to destroy the SwapChain. If I skip dropping (or leak) the winit::window::Window, the hang does not occur.

I think that Bevy might be violating the WGPU surface docs on this, from here

raw_window_handle must remain valid until after the returned Surface is dropped.

But Bevy will always destroy the winit Window before destroying the Surface.

It isn't clear why other drivers handle this more gracefully.

nicopap commented 1 year ago

Known issue. I've tried several times to solve it, but stopped, since for me, since the pipelined rendering change, it stopped happening altogether. See the already opened issues:

If it is really what you describe, I'm hyped. All I can say.

robojeb commented 1 year ago

Thanks for bringing together all the prior issues.

It heartens me that #839 and #1908 have stack-traces which show the stall occurring during swap-chain destruction. It makes it more likely we are actually looking at the same issue across all these machines. (It looks like it may even affect the intel driver as well).

5524 doesn't specifically mention the swap-chain but does occur while the surface is being unconfigured.

robojeb commented 1 year ago

I built and installed the amdvlk driver with debug symbols to see if I could dig further.

The issue is most definitely caused by closing the winit Window prior to shutting down the wgpu Surface in the renderer.

Internally the amdvlk driver spawns a background thread to handle presenting images to whatever the display back-end happens to be. In FIFO mode, after every frame presented this background thread will wait for the X-server to send an XCB_PRESENT_COMPLETE_NOTIFY by calling xcb_wait_for_special_event(). Because the window was closed, this never happens (see similar issue here).

Because the background thread never advances in its loop, it fails to process a Notify event that the Bevy render thread sends while tearing down the Swap-chain which sets a timeout for UINT32_MAX seconds (so technically Bevy will eventually close... in 136 years).

So the root cause is: 

Bevy --[blocks on]--> amdvlk worker --[waiting for event from]--> Xorg

While it could be argued that the amdvlk driver is doing something not very resilient, I do think that Bevy should fix the Window-Surface close order. This will make Bevy more resilient, and will comply with the wgpu documentation.

I put together a prototype which stores an Atomic token in the main app World to back-channel information from the Render World about the state of the wgpu Surface.

This is the most minimal change that I could think of, but is somewhat of a hack. If someone knows a better way to back-channel information from the Render World, or can move ownership of the Surface to the Main App World that might be a better solution, but I couldn't think of a way that didn't involve introducing heavy new dependencies into one of the crates.

Edit: I think this issue should be tagged with S-Needs-Design

Friz64 commented 6 months ago

13236 fixes this.