Sending ray from custom viewport

[ThomasAlban]

I'm creating an editor in bevy using egui_dock and want to add my own mouse picking. My camera renders to an image which is displayed in a widget. How should I make this work with bevy_mod_raycasting? I have tried scaling the mouse position so that 0,0 is the top left of the viewport and the window resolution is the bottom right of the viewport then sending that scaled mouse position to bevy_mod_raycast, but it's off by some scale factor and I can't work out why. I have checked the numbers and the maths is all correct. Is there something I'm missing or a better way of doing this? Thanks

[ThomasAlban]

I found a hacky way of doing it in the update_raycast_with_cursor system:

// ratio between viewport size and window size
let ratio = viewport_rect.size() / window_rect.size();

// scale the mouse pos so that top left of the window becomes top left of the viewport
// and bottom right of the window becomes bottom right of the viewport
let mut scaled_mouse_pos = window_rect.min
    + (mouse_pos - viewport_rect.min) * (window_rect.size() / viewport_rect.size());

scaled_mouse_pos *= ratio;
scaled_mouse_pos = scaled_mouse_pos.clamp(Vec2::ZERO, viewport_rect.max);

//grab the most recent cursor event if it exists
for mut pick_source in &mut query {
    pick_source.cast_method = RaycastMethod::Screenspace(scaled_mouse_pos);
}

I am wondering if there is a better way of doing it than this.

[aevyrie]

The screenspace method is really only intended for simple cases. You can use the transform casting method, or use the new immediate mode api to do this on the fly.

[darkwater]

I think this has to do with DPI scaling? With egui_dock and bevy_mod_picking, my cursor seems offset but only on my built-in 2x DPI display, and it works fine if I move the window to my external 1x DPI display.

It looks like Ray3d::from_screenspace() does take viewport offsets into account, but uses the wrong position:

pub fn from_screenspace(
    cursor_pos_screen: Vec2,
    camera: &Camera,
    camera_transform: &GlobalTransform,
) -> Option<Self> {
    let mut viewport_pos = cursor_pos_screen;
    if let Some(viewport) = &camera.viewport {
        viewport_pos -= viewport.physical_position.as_vec2();
    }
    camera
        .viewport_to_world(camera_transform, viewport_pos)
        .map(Ray3d::from)
}

Changing it to use logical coords fixes the issue for me:

if let Some(viewport) = &camera.logical_viewport_rect() {
    viewport_pos -= viewport.min;
}

Although, maybe cursor_pos_screen should be translated elsewhere instead? Either way, it seems this would be useful to fix in either bevy_mod_raycast or bevy_mod_picking.