dantman
commented
4 years ago I have plans for something similar, but I don't think I'm going to do things this exact way. i.e. I don't think I'll allow you to use a Touch controller without holding the controllers.
On the topic of using the overlay with a HOTAS you can check out the issues in the Hybrid HOTAS usage. Issues there are ones related to using the virtual overlay with a physical HOTAS. I'm tracking them and would like to do them in the future, however I likely won't work on them till I've finished all the other things needed for a 1.0 release for virtual controls. i.e. Full playability (full coverage of game features so there's nothing you need to drop your controllers and fallback to a mouse/keyboard/gamepad to do) and the QoL functionality needed to make the overlay comfortable to use.
The first step will be adding skeletal input support (#88). Controllers with partial finger tracking (i.e. Index controllers) will switch to a different mode similar to what you describe. Where you can make a pointing gesture with just your index finger and press buttons by touching them like you describe and pinch buttons in edit mode to grab and move them around. However this will require a pointing gesture to press buttons so it will not work when you're not actually holding the controller.
After that is done I may consider supporting non-controller hand tracking systems like Leapmotion (#89). When that's done you'll be able to drop your controllers entirely and instead just use your real tracked hands to press buttons when you're using your HOTAS.
Of a related note important to you. Oculus is going to be providing a hand tracking SDK for Quest. However it's limited to Quest applications (i.e. not PC games you play with Oculus Link). If they bring this feature to the PC SDK (Rift S/Oculus Link) then I'd consider supporting it.
However, in order to do that I would need to include the Oculus SDK and make a native Oculus overlay instead of a SteamVR overlay.
- This of course hinges on whether this is at all possible. I have never heard of anyone creating one and the Oculus documentation is not clear on whether this is possible. The API does allow compositing 2d quads like the rectangular SteamVR overlays we use. However it is not clear whether an application other than the game itself is allowed to submit those to the Oculus compositor while a different application (the game) is rendering a scene.
- Building a hybrid SteamVR/Oculus overlay application is also significantly more complex than just adding joystick axis actions that you can bind the Oculus Touch joysticks to. So adding native support like this will absolutely require I have permanent access to Oculus hardware (Rift S or Quest) in order to develop that native support and continue to test the overlay.
But, if all of those problems go away and the issues are dealt with at some point. Then just maybe, there is a chance you'll be able to use your HOTAS controllers and press buttons with your hands without needing to strap controllers to your hands or get extra hardware.
jonnyapps
Sorry, I don't use GitHub - this isn't an 'issue' but a question or a feature request maybe? Feel free to delete if I'm in the wrong place.
Is it possible to press the overlay buttons (retract hardpoints etc) just by gesturing a tap or pushing 'through' them instead of pressing trigger as well?
I'm using an Oculus Quest and could easily strap the controllers to my wrist, thus freeing my hands to use a physical joystick/hotas, but would love to then reach out to tap the button overlays.
Thanks.