Hit testing for gizmos

[pcwalton]

What problem does this solve or what need does it fill?

In many editors, such as Blender, gizmos are clickable. I want this for a particle system editor, where I show gizmos, and it feels weird to not have them be clickable.

What solution would you like?

I think an optional immediate mode hit test parameter for drawing gizmos (or perhaps global state with a config option, or a return value) would fit in well with the gizmos API. When you draw a gizmo, you can specify a 2D point that will be hit tested. If the hit test succeeds, the gizmo returns the 3D point that the user clicked on. The user shouldn't have to click exactly on a line; within some 2D distance would be acceptable.

An obvious question is what to do when the gizmo is covered up by a mesh. In that case the app can use bevy_mod_picking or similar to do the hit test as well. If both the gizmo and bevy_mod_picking test succeed, then process the event corresponding to the point nearer to the camera. This is why I propose that hit testing gizmos return the point at which they were clicked.

What alternative(s) have you considered?

Another possibility would be to make it not immediate mode and just create phantom meshes so that bevy_mod_picking could be used, but this seems cumbersome.

[BD103]

Linking to some discussion on Discord related to this :)

[pablo-lua]

I think the other way to proceed with this is related to https://github.com/bevyengine/bevy/issues/9178

[viridia]

I plan to implement clickable gizmos as part of my game editor, but likely they will not be based on Bevy's gizmo framework. The existing gizmos are great for debugging, but I'm not sure they are a sound basis for interactive 3d widgets.

My plan is to create a set of reactive components that can render 3d translucent shapes. These can then leverage bevy_mod_picking for input, and use bevy_reactor's reactivity for responding to changes in game state.