Impossible to change pointer's raycast layer mask at runtime

KodWhat commented 2 years ago

Describe the bug

I've upgraded my project from MRTK 2.7 to 2.8.2 to fix multiple issues, including one where the grab pointer did not account for layer mask changes at runtime (#10414)

However, since the upgrade that reworked the pointers and added per-cursor configuration for the layer mask (#10508), I can't change that layer mask at runtime anymore for any of the pointer declared.

To reproduce

Steps to reproduce the behavior:

Change the layer mask of the pointer profile at runtime using the exact same line as we used in 2.7
```
CoreServices.InputSystem.InputSystemProfile.PointerProfile.PointingRaycastLayerMasks[0].value = layerMask;
```
=> The multiple pointer options do not reflect the change

Change the layer mask for every pointer option at runtime using

foreach (PointerOption pointer in CoreServices.InputSystem.InputSystemProfile.PointerProfile.PointerOptions)
{
pointer.PrioritizedLayerMasks[0].value = layerMask;
}

=> The option seems to be updated but is not reflected to the instantiated pointer

Expected behavior

The change in layer mask at runtime should be applied to any instantiated pointer on the scene

Your setup (please complete the following information)

Unity Version: 2019.4.39f1
MRTK Version: 2.8.2

Target platform (please complete the following information)
HoloLens 2

Additional context

Changing the layer mask at runtime is a mandatory feature for us. It allows us to enable spatial mapping collisions on or off and disable interaction with other objects when showing a modal window

AMollis commented 2 years ago

In another thread I suggested that instead of modifying the MRTK profile at runtime, try modifying the pointer's PrioritizedLayerMasksOverride.

        /// <summary>
        /// The physics layers to use when performing scene queries.
        /// </summary>
        /// <remarks>If set, will override the <see cref="IMixedRealityInputSystem"/>'s default scene query layer mask array.</remarks>
        /// <example>
        /// Allow the pointer to hit SR, but first prioritize any DefaultRaycastLayers (potentially behind SR)
        /// <code language="csharp"><![CDATA[
        /// int sr = LayerMask.GetMask("SR");
        /// int nonSR = Physics.DefaultRaycastLayers &amp; ~sr;
        /// IMixedRealityPointer.PrioritizedLayerMasksOverride = new LayerMask[] { nonSR, sr };
        /// ]]></code>
        /// </example>
        LayerMask[] PrioritizedLayerMasksOverride { get; set; }

Setting PrioritizedLayerMasksOverride appears to have unblocked this scenario.

@KodWhat have you tried setting the pointer's PrioritizedLayerMasksOverride?

KodWhat commented 2 years ago

From the quick research and observation through the inspector that I've done, this solution has an issue. A pointer is not instantiated until needed. That would mean that if a user only uses one hand when we set the override, the other hand would not get that override and will be able to interact using the default layer mask.

When exploring the scene at play time, no pointer is visible at first, then, if I enable a hand, one pointer is created. If I decide to disable this hand and enable the other side, the same pointer will get reused. When both hands are enabled, two pointers are created but when trying to enable only one after that, the last one gets reused, no matter the hand.

It lacks consistency and I can't really consider this solution as reliable

RogPodge commented 2 years ago

Hey there, sorry for the delayed response.

The pointer profile is intended to be used like a "template". Changes made to the pointer profile were not meant to be reflected onto pointers which already exist in the scene. If you need to change the raycast layermask of an in-scene pointer at runtime, we suggest you change that pointer's PrioritizedLayerMasksOverride as described by AMolis.

There were and still are a lot of inconsistencies with how the profile's interact with objects in the scene, due in part to the ever expanding scope of it and how long it's been developed. With 2.8 we tried as much as possible to make the pointer profile the equivalent of "instantiation settings", but understandably, because pointers are spawned and recycled when needed, it doesn't feel like it always follows this paradigm.

When exploring the scene at play time, no pointer is visible at first, then, if I enable a hand, one pointer is created. If I decide to disable this hand and enable the other side, the same pointer will get reused. When both hands are enabled, two pointers are created but when trying to enable only one after that, the last one gets reused, no matter the hand.

It lacks consistency and I can't really consider this solution as reliable

This isn't something that is easily addressible with the current state of MRTK2's resource pooling. I would suggest having a script on your "hand" object which keeps track of the pointer that gets assigned the same "handedness", and then making sure that the PrioritizedLayerMasksOverride is set correctly.

All-in-all, we intended to simplify the pointer workflow by keeping the pointers themselves as the sole source of truth, rather than splitting the responsibility with the pointer profile. We understand that this has caused some inconveniences with pre-exisitng workflows, but we thought that this breaking change was needed in order to make MRTK2 more understandable and easy to modify in the future.

IssueSyncBot commented 8 months ago

We appreciate your feedback and thank you for reporting this issue.

Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.

Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.

Please note that MRTK3 was released in August 2023. It features an all-new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visit https://www.mixedrealitytoolkit.org.

Thank you for your continued support of the Mixed Reality Toolkit!

microsoft / MixedRealityToolkit-Unity