Open musjj opened 5 months ago
I am also having the same issue. Using the same exact code I cannot get any pointers to work. When using the noisy debugger, It's obvious that It sees the Pixel Perfect camera as a large texture and not an actual ui element. So the target Camera is the outer camera. Not sure how to fix.
If you are trying to pick into a texture, then you need to add a pointer to that render target. https://github.com/aevyrie/bevy_mod_picking/pull/327
This all comes back to how the picking pipeline works: https://docs.rs/bevy_mod_picking/latest/bevy_mod_picking/#the-picking-pipeline
All you need to do is inform the backends that there is a pointer located at some x/y position on the target, and they can handle everything else automatically. Doing this correctly will probably require raycasting: raycast onto the render-to-texture quad, find the uv coordinate that the pointer is over, and send a PointerMove
of that pointer over that render target.
So in that example they aren't using raycasting. Is there not a way to simply attach the current pointer to the original in game camera? As opposed to translating the positions of the mouse to the render texture? Apologies as I'm a new to bevy and trying to learn a bit more about how it works.
That won't work. The pointer's position on the window is not the same as the pointer's position on the lower resolution render target. For example, if the window is 1920x1080, and you put the pointer in the bottom right, the pointer's position on the low resolution target is not (1920,1080), it depends on the size of that render target and its position on screen. If it is half resolution, the position 1920, 1080 on the window would map to 960, 540 on the half resolution render target. However, this ignores that the main camera could move around such that the render target does not fill the window, in which case you also need to compute this offset. You could hardcode this and assume the render target always fills the screen and has some fixed resolution ratio, but it might be brittle.
Thank you for the write up, that makes sense. Perhaps rendering to a texture isn’t the greatest method for achieving this? Maybe there are better methods for achieving pixel perfect rendering?
I don't think there is anything wrong with this approach, it's just not immediately obvious how to handle this in a robust way. Considering this is 2D-only, it shouldn't be too hard though. All you need is the position of the pointer inside the texture. If it is in bevy_ui, I think it already computes that for you. If you are doing this manually with two 2d cameras, that also shouldn't be too hard. The only steps you need to follow are:
InputMove
event for that pointer, with the quad's RenderTarget
and the position computed in step 1This is very helpful, I appreciate you taking the time to do this! Hopefully this will help others as well.
Here's a minimal reproduction code. It's a combination of 2d/pixel_grid_snap.rs and ui/button.rs from the official examples:
Expand
```rust use bevy::{ prelude::*, render::{ camera::RenderTarget, render_resource::{ Extent3d, TextureDescriptor, TextureDimension, TextureFormat, TextureUsages, }, view::RenderLayers, }, window::WindowResized, }; use bevy_mod_picking::prelude::*; /// In-game resolution width. const RES_WIDTH: u32 = 160; /// In-game resolution height. const RES_HEIGHT: u32 = 90; /// Default render layers for pixel-perfect rendering. /// You can skip adding this component, as this is the default. const PIXEL_PERFECT_LAYERS: RenderLayers = RenderLayers::layer(0); /// Render layers for high-resolution rendering. const HIGH_RES_LAYERS: RenderLayers = RenderLayers::layer(1); fn main() { App::new() .add_plugins(DefaultPlugins.set(ImagePlugin::default_nearest())) .add_plugins(DefaultPickingPlugins) .insert_resource(Msaa::Off) .insert_resource(DebugPickingMode::Normal) .add_systems(Startup, (setup_camera, setup_ui).chain()) .add_systems(Update, fit_canvas) .run(); } /// Low-resolution texture that contains the pixel-perfect world. /// Canvas itself is rendered to the high-resolution world. #[derive(Component)] struct Canvas; /// Camera that renders the pixel-perfect world to the [`Canvas`]. #[derive(Component)] struct InGameCamera; /// Camera that renders the [`Canvas`] (and other graphics on [`HIGH_RES_LAYERS`]) to the screen. #[derive(Component)] struct OuterCamera; /// Our button #[derive(Component)] struct MyButton; fn setup_ui(mut commands: Commands, query: QueryI tried to read through the UI backend and I think this is the problem. We have two cameras:
The first one targets an image handle, while the second one targets the main window. The UI node is associated with the first camera, while the main pointer is associated with the second camera.
The
bevy_ui
backend looks for pointers associated with the camera that the UI node is rendering to, before testing if those pointers intersects with the UI node. So our UI here is always skipped because our UI camera has no associated pointers.It seems that the sprite backend does not have this problem. I think it's because the sprite backend does not care to which camera the sprite is getting rendered to.
Do you think that there is a general way to solve this problem upstream? Some pointers would be appreciated.