Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
130 stars 2 forks source link

Raycast to UIX elements #1458

Open 5H4D0W-X opened 5 months ago

5H4D0W-X commented 5 months ago

Is your feature request related to a problem? Please describe.

Raycasts can currently not interact with the elements on UIX canvases, instead they just hit the BoxCollider of the Canvas.

Describe the solution you'd like

A specific set of nodes and components to enable UIX raycasting. If the raycast hits the collider of a canvas, it checks what UIX rect it should be hitting and returns the RectTransform of the hit element.

Describe alternatives you've considered

Using a color sampler and flashing each rect like a light pen, but that’s ridiculous and doesn’t work on any arbitrary canvas.

Additional Context

Originally requested by @ KacperLaska on YouTube for accessibility purposes, as this would allow for tooltips that read the contents of a UIX element aloud using dynvars.

Requesters

ShadowX | shdw_x

epicEaston197 commented 5 months ago

This would essentially also be another step to be able to select UIX elements with your devTool since that's also just a Raycast

Banane9 commented 5 months ago

Ah yes, I've sorta implemented that on the mod side for BoundedUIX, but that of course won't work through Protoflux.

Having a more general way to have any Text you point/click at read to you would sound like a very powerful accessibility feature.

lxw404 commented 5 months ago

This would be extremely useful for accessibility and would also make it easier to enable hover popups for additional help UI.

epicEaston197 commented 5 months ago

People have mentioned accessibility around this feature so oftenly I think the accessibility tag is warranted here

989onan commented 5 months ago

same should have accessibility

epicEaston197 commented 5 months ago

This feature isn't necessarily an accessibility feature but the things that could be built around it would be an accessibility feature

5H4D0W-X commented 5 months ago

If the (constant) raycaster has a user input, it could even be used to trigger hover and press events. This could make it possible to build non-laser UI with UIX, for example with a touchpad-operated cursor or eye tracking. This might be too insecure though as it would make it super simple to trigger button presses on other people

JackTheFoxOtter commented 5 months ago

If the (constant) raycaster has a user input, it could even be used to trigger hover and press events. This could make it possible to build non-laser UI with UIX, for example with a touchpad-operated cursor or eye tracking. This might be too insecure though as it would make it super simple to trigger button presses on other people

It is already super simple to trigger in-world button presses for other people. As long as that won't work in userspace (from outside of userspace), I don't see any additional security concern here.

shiftyscales commented 5 months ago

Having text to speech for interacting with UI elements/some mechanism for communicating what elements in a canvas are hovered over should be its own issue separate from this.