TAA (was: panic when taa is enabled)

[LaoLittle]

app.add_plugins((DefaultPlugins, TemporalAntiAliasPlugin, OutlinePlugin));

commands.spawn((Camera3dBundle {
        transform: Transform::from_xyz(-2.0, 2.5, 5.0).looking_at(Vec3::ZERO, Vec3::Y),
        ..default()
    }, TemporalAntiAliasBundle::default()));

commands
        .spawn(PbrBundle {
            mesh: meshes.add(Mesh::from(Cube { size: 1.0 })),
            material: materials.add(Color::rgb(0.1, 0.1, 0.9).into()),
            transform: Transform::from_xyz(0.0, 1.0, 0.0),
            ..default()
        })
        .insert(OutlineBundle {
            outline: OutlineVolume {
                visible: true,
                colour: Color::rgba(0.0, 1.0, 0.0, 1.0),
                width: 25.0,
            },
            ..default()
        });

2023-11-20T11:41:19.129406Z ERROR log: Handling wgpu errors as fatal by default    
thread '<unnamed>' panicked at 'wgpu error: Validation Error

Caused by:
    In a RenderPass
      note: encoder = `<CommandBuffer-(0, 1, Vulkan)>`
    In a draw command, indexed:true indirect:false
      note: render pipeline = `outline_pipeline`
    The pipeline layout, associated with the current render pipeline, contains a bind group layout at index 0 which is incompatible with the bind group layout associated with the bind group at 0

', C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\backend\direct.rs:3056:5
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library\core\src\panicking.rs:67
   2: wgpu::backend::direct::default_error_handler
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\backend\direct.rs:3056
   3: core::ops::function::FnOnce::call_once<void (*)(enum2$<wgpu::Error>),tuple$<enum2$<wgpu::Error> > >
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ops\function.rs:79
   4: core::ops::function::FnOnce::call_once<void (*)(enum2$<wgpu::Error>),tuple$<enum2$<wgpu::Error> > >
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ops\function.rs:79
   5: alloc::boxed::impl$49::call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\alloc\src\boxed.rs:2007
   6: wgpu::backend::direct::ErrorSinkRaw::handle_error
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\backend\direct.rs:3042
   7: wgpu::backend::direct::Context::handle_error<wgpu_core::command::render::RenderPassError>
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\backend\direct.rs:324
   8: wgpu::backend::direct::impl$7::command_encoder_end_render_pass
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\backend\direct.rs:1946
   9: wgpu::context::impl$5::command_encoder_end_render_pass<wgpu::backend::direct::Context>
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\context.rs:2796
  10: wgpu::impl$54::drop
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\wgpu-0.17.2\src\lib.rs:3898
  11: core::ptr::drop_in_place
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ptr\mod.rs:497
  12: core::ptr::drop_in_place<bevy_render::render_phase::draw_state::TrackedRenderPass>
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ptr\mod.rs:497
  13: bevy_learn::outline::node::impl$7::run
             at .\src\outline\node.rs:233
  14: bevy_render::renderer::graph_runner::RenderGraphRunner::run_graph
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_render-0.12.0\src\renderer\graph_runner.rs:190
  15: bevy_render::renderer::graph_runner::RenderGraphRunner::run_graph
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_render-0.12.0\src\renderer\graph_runner.rs:197
  16: bevy_render::renderer::graph_runner::RenderGraphRunner::run<bevy_render::renderer::render_system::closure_env$1>
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_render-0.12.0\src\renderer\graph_runner.rs:63
  17: bevy_render::renderer::render_system
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_render-0.12.0\src\renderer\mod.rs:33
  18: core::ops::function::FnMut::call_mut
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ops\function.rs:166
  19: core::ops::function::impls::impl$3::call_mut
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\ops\function.rs:294
  20: bevy_ecs::system::exclusive_function_system::impl$2::run::call_inner
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\system\exclusive_function_system.rs:207
  21: bevy_ecs::system::exclusive_function_system::impl$2::run
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\system\exclusive_function_system.rs:210
  22: bevy_ecs::system::exclusive_function_system::impl$1::run<void (*)(),void (*)(ref_mut$<bevy_ecs::world::World>)>
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\system\exclusive_function_system.rs:107
  23: core::panic::unwind_safe::impl$26::poll<enum2$<bevy_ecs::schedule::executor::multi_threaded::impl$3::spawn_exclusive_system_task::async_block_env$1> >
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\panic\unwind_safe.rs:296
  24: futures_lite::future::impl$14::poll::closure$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:626
  25: core::panic::unwind_safe::impl$23::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\panic\unwind_safe.rs:271
  26: std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panicking.rs:500
  27: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<futures_lite::future::impl$14::poll::closure_env$0<core::panic::unwind_safe::AssertUnwindSafe<enum2$<bevy_ecs::schedule::executor::multi_threaded::im
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panicking.rs:464
  28: std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panic.rs:142
  29: futures_lite::future::impl$14::poll
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:626
  30: async_executor::impl$5::spawn::async_block$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\async-executor-1.6.0\src\lib.rs:151
  31: async_task::raw::RawTask<enum2$<async_executor::impl$5::spawn::async_block_env$0<enum2$<core::result::Result<tuple$<>,alloc::boxed::Box<dyn$<core::any::Any,core::marker::Send>,alloc::alloc::Global> > >,futures_lite::future::CatchUnwind<core::panic::unwind
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\async-task-4.5.0\src\raw.rs:568
  32: async_executor::impl$5::tick::async_fn$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\async-executor-1.6.0\src\lib.rs:216
  33: bevy_tasks::thread_executor::impl$2::tick::async_fn$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\thread_executor.rs:105
  34: bevy_tasks::task_pool::impl$2::execute_scope::async_fn$0::async_block$0::async_block$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:512
  35: core::panic::unwind_safe::impl$26::poll<enum2$<bevy_tasks::task_pool::impl$2::execute_scope::async_fn$0::async_block$0::async_block_env$0<tuple$<>,enum2$<bevy_tasks::task_pool::impl$2::scope_with_executor_inner::async_block$0::async_block_env$0<bevy_ecs::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\panic\unwind_safe.rs:296
  36: futures_lite::future::impl$14::poll::closure$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:626
  37: core::panic::unwind_safe::impl$23::call_once
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\core\src\panic\unwind_safe.rs:271
  38: std::panicking::try::do_call
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panicking.rs:500
  39: std::panicking::try
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panicking.rs:464
  40: std::panic::catch_unwind
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\panic.rs:142
  41: futures_lite::future::impl$14::poll
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:626
  42: bevy_tasks::task_pool::impl$2::execute_scope::async_fn$0::async_block$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:515
  43: futures_lite::future::impl$12::poll<alloc::vec::Vec<tuple$<>,alloc::alloc::Global>,enum2$<bevy_tasks::task_pool::impl$2::execute_scope::async_fn$0::async_block_env$0<tuple$<>,enum2$<bevy_tasks::task_pool::impl$2::scope_with_executor_inner::async_block$0::
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:526
  44: bevy_tasks::task_pool::impl$2::execute_scope::async_fn$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:518
  45: bevy_tasks::task_pool::impl$2::scope_with_executor_inner::async_block$0<bevy_ecs::schedule::executor::multi_threaded::impl$2::run::closure_env$1,tuple$<> >
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:429
  46: futures_lite::future::block_on::closure$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:89
  47: std::thread::local::LocalKey<core::cell::RefCell<tuple$<parking::Parker,core::task::wake::Waker> > >::try_with
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\thread\local.rs:270
  48: std::thread::local::LocalKey<core::cell::RefCell<tuple$<parking::Parker,core::task::wake::Waker> > >::with<core::cell::RefCell<tuple$<parking::Parker,core::task::wake::Waker> >,futures_lite::future::block_on::closure_env$0<alloc::vec::Vec<tuple$<>,alloc::
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\thread\local.rs:246
  49: futures_lite::future::block_on
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\futures-lite-1.13.0\src\future.rs:79
  50: bevy_tasks::task_pool::TaskPool::scope_with_executor_inner<bevy_ecs::schedule::executor::multi_threaded::impl$2::run::closure_env$1,tuple$<> >
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:383
  51: std::thread::local::LocalKey<alloc::sync::Arc<bevy_tasks::thread_executor::ThreadExecutor> >::try_with
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\thread\local.rs:270
  52: std::thread::local::LocalKey<alloc::sync::Arc<bevy_tasks::thread_executor::ThreadExecutor> >::with<alloc::sync::Arc<bevy_tasks::thread_executor::ThreadExecutor>,bevy_tasks::task_pool::impl$2::scope_with_executor::closure_env$0<bevy_ecs::schedule::executor
             at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be\library\std\src\thread\local.rs:246
  53: bevy_tasks::task_pool::TaskPool::scope_with_executor
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_tasks-0.12.0\src\task_pool.rs:316
  54: bevy_ecs::schedule::executor::multi_threaded::impl$2::run
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\schedule\executor\multi_threaded.rs:198
  55: bevy_ecs::world::impl$3::try_run_schedule::closure$0
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\world\mod.rs:2074
  56: bevy_ecs::world::World::try_schedule_scope<tuple$<>,bevy_utils::intern::Interned<dyn$<bevy_ecs::schedule::set::ScheduleLabel> >,bevy_ecs::world::impl$3::try_run_schedule::closure_env$0<bevy_utils::intern::Interned<dyn$<bevy_ecs::schedule::set::ScheduleLab
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\world\mod.rs:2007
  57: bevy_ecs::world::World::schedule_scope
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\world\mod.rs:2059
  58: bevy_ecs::world::World::run_schedule
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_ecs-0.12.0\src\world\mod.rs:2088
  59: bevy_app::app::SubApp::run
             at C:\Users\LaoLittle\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bevy_app-0.12.0\src\app.rs:166
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

[komadori]

I can reproduce, but I'm not sure why it occurs yet. Bind group 0 should be the mesh view bindings, which doesn't look like it's affected by TAA.

[komadori]

I've fixed the panic with bf2385f167f903be1bc821cc44bad382225a2f0b, but for TAA to work properly the outline shader also needs to write to the depth and motion vector textures. Alternatively, the outline pass could run after TAA but then outlines wouldn't be anti-aliased.

[LaoLittle]

but for TAA to work properly the outline shader also needs to write to the depth and motion vector textures. Alternatively, the outline pass could run after TAA but then outlines wouldn't be anti-aliased.

I think the TAA just works.

Without TAA:

With TAA enabled:

[komadori]

Is the sphere moving in your example? If it's not moving then the motion vectors will be zero just like the background and it will look correct. However, if you enable TAA on one of the examples with moving objects, then you can see that the border of the outline has a fuzzy effect. Per comments in the Bevy source code:

It is very important that correct motion vectors are written for everything on screen. Failure to do so will lead to ghosting artifacts. For instance, if particle effects are added using a third party library, the library must either:

1. Write particle motion vectors to the motion vectors prepass texture
2. Render particles after TAA

I will address this issue, but I cannot guarantee a time scale.

[LaoLittle]

Is the sphere moving in your example?

No. You are right, the borders of moving objects are kinda fuzzy

[Shatur]

@komadori could you draft a patch release with the panic fix?

[komadori]

@Shatur I've released the interim fix in 0.6.1.

[komadori]

One problem is that TAA reads from both the motion vector buffer and the depth buffer. However, the outline pass reuses and clears the depth buffer, and so I'm not sure how to make TAA work correctly after outlining.

Presently, I've changed the render graph so that the outlining node runs between the tone-mapping and FXAA labels and hence after TAA. This means that there are no TAA artefacts caused by outlining, but outlines are not anti-aliased by TAA.

The new ui_aa example on master illustrates this.

[Shatur]

@komadori panics for me with TAA enabled on 0.14 :(

thread '<unnamed>' panicked at /home/gena/.cargo/registry/src/index.crates.io-6f17d22bba15001f/wgpu-0.20.1/src/backend/wgpu_core.rs:2996:5:
wgpu error: Validation Error

Caused by:
    In a RenderPass
      note: encoder = `<CommandBuffer-(16, 1, Vulkan)>`
    In a draw command, indexed:true indirect:false
      note: render pipeline = `outline_pipeline`
    Incompatible bind group at index 1 in the current render pipeline
      note: Should be compatible an with an explicit bind group layout with label = `skinned_mesh_layout`
      note: Assigned explicit bind group layout with label = `skinned_motion_mesh_layout`
      note: Entry 6 not found in expected bind group layout

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace