Neos-Metaverse / NeosPublic

A public issue/wiki only repository for the NeosVR project
195 stars 9 forks source link

Make UI Hand-Tracking/Leapmotion friendly #1456

Open InsaneGrox opened 3 years ago

InsaneGrox commented 3 years ago

So I was thinking about the leapmotion support and discussing it with PeterTheThinker, and I thought to myself that motion controllers should only be a neccessity for development and specific interactions that require them by hardware, especially considering AR hardware is unlikely to ship with motion controllers as they will likely use hand tracking instead, so I think a good first step would be making UI interactable with fingers and hand gestures.

The UI would essentially act as a touchscreen, where you can touch it with your finger to interact with it, allowing for use of the dash without any controllers, this would preferrably also have the dash close enough to just reach out and touch, optimally the dash would act like a touch-sensitive smartphone, including scrolling and other touch gestures.

As a bonus, a pinch-style laser while using hand tracking would also possibly help for laser interactions. (simular to how oculus does it's hand tracking)

also UI opening could be done this way, IE: Dash could be opened by touching your thumb and your pinky until a circle fills up, you can close it again by doing the same thing. Radial could be opened by touching your thumb and your ring finger until a circle fills up, once it's opened, you touch the option you want with your index finger, or you hold your thumb and ring finger together longer and move your hand in accordance with your desired option (kinda like holding the trigger on a controller while the radial is open)

shiftyscales commented 3 years ago

Considerations have already been made towards this, e.g. in the separation of 'passive' and 'active' touches per-device.

The touch is now being separated to "active" and "passive". Active touches will interact with UI and items by default, passive won't.

The reason for this is that with things like Leap Motion and other finger tracking it's impossible to interact with most UI's, so those will now be considered active touches, while when using controllers those will be passive.

It's not presently a high priority, but will be much more of one when AR hardware becomes more mainstream/adopted.