search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6.01k stars 2.12k forks source link

ObjectManipulator should be more general to support non-hand inputs #7274

Closed adambehringer closed 2 years ago

adambehringer commented 4 years ago

Describe the problem

I would like to scale, rotation, and translate objects using inputs besides hands (such as a game pad). If the object has a ManipulationHandler, it would seem logical to use it to perform the transforms so that they will obey the active constraints, and also to trigger related events (OnManipulationStarted, ...).

Currently ManipulationHandler contains hard-coded logic specific to one and two-handed gestures and does not provide public methods for applying transforms through other inputs.

Describe the solution you'd like

In my opinion ManipulationHandler should be more general, handling the application of transforms to an object along with the related constraints and side effects. The logic related to specific input controls (such as hands) should be modular and extensible, since transforms could be triggered by anything including game pads, voice commands, or other input methods.

Describe alternatives you've considered

I have considered manipulating object transforms directly. Unfortunately, this bypasses the pre-existing constraints and events in the manipulation handler.

adambehringer commented 4 years ago

Note that this has accessibility implications as well.

Alexees commented 4 years ago

Related #7104

polar-kev commented 4 years ago

ObjectManipulator replaces ManipulationHandler as of 2.4.0 but this issue is still relevant.

david-c-kline commented 3 years ago

This looks to be for non-VR / non-Hand controls (ex: xbox controller). Things to consider:

adambehringer commented 3 years ago

Generalizing the behavior of ObjectManipulator would aid the integration of controller such as the Xbox Controller. However, it would also be useful for things like speech commands ("scale up the blue box by 50%") or indirect controls such as a holographic control panel that manipulates 3D objects in the scene.

In both those cases, selecting specific handles is probably not part of the user flow. So I would suggest that manipulations should be accessible directly through an API, and not exclusively through the manipulations widgets. I'd still want the API calls to be limited by the manipulation constraints though. Part of the issue with bypassing ObjectManipulator is missing out on those constraints.

RogPodge commented 3 years ago

This is an excellent suggestion and we'll make sure to consider it for the next evolution of our MRTK UI tools!

RogPodge commented 2 years ago

This is officially a reality with MRTK v3. Look forward to using the object manipulator with controllers, eye gaze and more in the upcoming version.

Controller support still pending but the hooks for it are reality available.

Zee2 commented 2 years ago

This is definitely a reality in MRTK3! ObjectManipulator works uniformly across all input modalities, including any custom interactors you might cook up yourself.