microsoft / MixedRealityDesignLabs_Unity

Mixed Reality Design Labs share experimental samples, explorations and learning from Windows Mixed Reality Design group. If you are looking for official toolkit, please use Mixed Reality Toolkit
MIT License
408 stars 110 forks source link

Clicker not responding the same as airtap when not pointing at object #99

Closed Zulex closed 6 years ago

Zulex commented 6 years ago

Hello All,

When trying to use the clicker I noticed something I can't explain the cause of. When pointing at UI buttons or objects, the clicker works exactly the same as Airtap.

Because for my application I need to be able to use the clicker when looking into blank space, I made a script which reads the bool "Select Button" from the script "InputShell" (Which is attached to InputMapping). See image 1. This bool is used instead of "onSelect" because it seemed the only working way of clicking into open space.

Now when staring at a blank space and using the clicker, nothing happens. When gazing at buttons, the clickers works fine. When staring at a blank space and using the airtap, my code now works. In image 2 you can see my script, which works fine for airtaps.

The reason I want to be able to use the clicker, is because person 1 is using the clicker and person 2 is wearing the HoloLens. Hope someone can explain why airtap is setting this bool to pressed and clicker not or suggest a better method.

Let me know if any information is missing!

Using Unity 2017.1.0f3

image

image

paseb commented 6 years ago

@Zulex,

Sorry for the late reply. I remember running into this some time back on a similar prototype and it was due to the clicker input being routed as mouse press events and getting ignored by a check that it was over an interactable object. We had a separate input mapping for the clicker itself but never published it as it was under the original clickers code name and inputs still routed. I'll take a look and see if I can find the part filtering out press events when not over an object.

thanks, -pat

Zulex commented 6 years ago

@paseb

Thanks for commenting and taking a look into it. If there's something I can do to help or speed up the process, please let me know. If there are other quick fixes for this, I'm also glad to hear this.

Thanks in advance and hope to hear soon from you!

paseb commented 6 years ago

Hi @Zulex, Unfortunately I don't have the spare cycles to get an update in for this or look for a quick fix but have and example clicker script you should be able to use to get clicker input separately (Note this is from an older codebase and will probably need some adjustments):

using UnityEngine;
using System;
using HUX;

#if UNITY_WSA
using UnityEngine.VR.WSA.Input;
#endif

public class InputSourceClicker : InputSourceBase, ITargetingInputSource
{
    public event Action<InputSourceBase, bool> OnSelectChanged = delegate { };
    public event Action<InputSourceBase, bool> OnMenuChanged = delegate { };

    public bool ButtonPressed;

    public float BloomGesture_ButtonHoldTime = 2f;

    public float ButtonHoldTime;
    public ButtonControlState menuGesture = new ButtonControlState();

    // Special logic for a hand vector while dragging
    public bool scrolling;
    Quaternion dragStartHeadRotation = Quaternion.identity;
    public Quaternion adjustedTargetRot = Quaternion.identity;
    public Vector2 targetRangeDegrees = new Vector2(45f, -45f);

    // For navigation events
#if UNITY_WSA
    GestureRecognizer gestureRecognizer;
#endif

    public class CurrentHandState
    {
        public uint HandId;

        public Vector3 Position;
        public Vector3 Velocity;

        public bool Pressed;
        public bool LastPressed;

        public double SourceLossRisk;
        public Vector3 SourceLossMitigationDirection;
    }

    // Targeting source interface
    bool ITargetingInputSource.ShouldActivate()
    {
        return ButtonPressed;
    }
    Vector3 ITargetingInputSource.GetTargetOrigin()
    {
        return Camera.main.transform.position;
    }
    Quaternion ITargetingInputSource.GetTargetRotation()
    {
        if (IsManipulating())
        {
            return adjustedTargetRot;
        }
        return Camera.main.transform.rotation;
    }

    bool ITargetingInputSource.IsSelectPressed()
    {
        return ButtonPressed;
    }

    bool ITargetingInputSource.IsMenuPressed()
    {
        return menuGesture.PressedOnce;
    }

    void ITargetingInputSource.OnActivate(bool activated)
    {

    }
    bool ITargetingInputSource.IsReady()
    {
        return true;
    }

    bool ITargetingInputSource.IsTargetingActive()
    {
        return true;
    }

    public override void _Update()
    {
        if (ButtonPressed)
        {
            ButtonHoldTime += Time.deltaTime;
        }
        else
        {
            ButtonHoldTime = 0;
        }
        bool wasPressed = menuGesture.pressed;
        menuGesture.ApplyState(ButtonHoldTime >= BloomGesture_ButtonHoldTime && InputShellMap.Instance.CanCompleteHoldGesture());

        if (wasPressed != menuGesture.pressed)
        {
            OnMenuChanged(this, menuGesture.pressed);
        }

        base._Update();
    }

    private void Awake()
    {
#if UNITY_WSA
        InteractionManager.SourcePressed += WSAFingerPressedEvent;
        InteractionManager.SourceReleased += WSAFingerPressedEvent;

        gestureRecognizer = new GestureRecognizer();
        gestureRecognizer.SetRecognizableGestures(GestureSettings.ManipulationTranslate);

        gestureRecognizer.ManipulationStartedEvent += GestureRecognizer_NavigationUpdateEvent;
        gestureRecognizer.ManipulationCompletedEvent += GestureRecognizer_NavigationUpdateEvent;
        gestureRecognizer.ManipulationCanceledEvent += GestureRecognizer_NavigationDoneEvent;
        gestureRecognizer.ManipulationUpdatedEvent += GestureRecognizer_NavigationDoneEvent;

        gestureRecognizer.StartCapturingGestures();
#endif
    }

#if UNITY_WSA
    private void GestureRecognizer_NavigationUpdateEvent(InteractionSourceKind source, Vector3 normalizedOffset, Ray headRay)
    {
        if (source == InteractionSourceKind.Controller)
        {
            if (!scrolling)
            {
                scrolling = true;
                dragStartHeadRotation = Veil.Instance.HeadTransform.rotation;
                normalizedOffset = Vector3.zero;
            }
            adjustedTargetRot = Quaternion.LookRotation(headRay.direction);
        }
    }

    private void GestureRecognizer_NavigationDoneEvent(InteractionSourceKind source, Vector3 normalizedOffset, Ray headRay)
    {
        if (source == InteractionSourceKind.Controller)
        {
            scrolling = false;
        }
    }
#endif

    Quaternion buildLocalTargetRot(float navX, float navY)
    {
        return Quaternion.Euler(navY * targetRangeDegrees.y, navX * targetRangeDegrees.x, 0);
    }

    Quaternion buildWorldTargetVector(float navX, float navY)
    {
        return dragStartHeadRotation * buildLocalTargetRot(navX, navY);
    }

#if UNITY_WSA
    private void OnDestroy()
    {
        InteractionManager.SourcePressed -= WSAFingerPressedEvent;
        InteractionManager.SourceReleased -= WSAFingerPressedEvent;
    }

    private void WSAFingerPressedEvent(InteractionSourceState state)
    {
        if (state.source.kind == InteractionSourceKind.Controller)
        {
            bool newState = ButtonPressed != state.pressed;
            ButtonPressed = state.pressed;
            if (ButtonPressed)
            {
                GestureRecognizer_NavigationUpdateEvent(InteractionSourceKind.Controller, Vector3.zero, new Ray());
            }

            if (newState)
            {
                OnSelectChanged(this, ButtonPressed);
            }
        }
    }
#endif
}

If this doesn't work look for any place where it's checking the focuser that there's a valid interactible before sending the event.

Hope this helps! -pat

Zulex commented 6 years ago

@paseb

Thank you so much! The code worked flawlessly, even when not staring at an object and the cursor turned off.