Closed provencher closed 9 months ago
@provencher, the demo build would be really cool for us to explore
@julenka @cre8ivepark for UX planning and prototyping around this. This is likely something we'd want to prototype internally before we ship the feature out.
@wiwei I'll see about getting you a build to sideload.
Here is a bit of code from our snapping cursor if (mOverCanvas) // cursor is on menu { // Raycast the canvas to see if the cursor is directly over a UI element. mPointerEventData.position = PointerManager.CameraInput.WorldToScreenSpace(cursorPosition);
mRaycastResults.Clear();
mMenuRaycaster.Raycast(mPointerEventData, mRaycastResults);
// pick a suitable result from the raycast hits
foreach (RaycastResult result in mRaycastResults)
{
FocusHandler focus = result.gameObject.GetComponentInParent<FocusHandler>();
if (focus == null || !focus.CanBeFocused()) continue;
newFocus = focus;
}
}
if (newFocus == null) // the initial raycast didn't find a focusable element under the cursor so try snapping to a nearby element
{
// Get all focusable objects if not already found
if (mFocusHandlers == null)
{
FindFocusables();
}
float closestDistance = mFocusedObject == null ? mStartFocusDistance : mLoseFocusDistance;
foreach (FocusHandler focusable in mFocusHandlers)
{
if (focusable != null && focusable.CanBeFocused())
{
focusable.FocusRectTransform.GetWorldCorners(mTempWorldConers);
// Compute the distance from the cursor to the edge of the UI elements
for (int i = 0; i < 4; i++)
{
Vector3 vertex1 = mTempWorldConers[i];
Vector3 vertex2 = mTempWorldConers[(i + 1) % 4];
float distance = GeometryUtils.DistanceFromEdgetoPoint(vertex1, vertex2, cursorPosition);
// bias towards the currently focused element, stabilizing the gaze
if (mFocusedObject != null && focusable != mFocusedObject)
{
distance += mFocusBias;
}
if (distance < closestDistance)
{
closestDistance = distance;
newFocus = focusable;
}
}
}
}
}
This sounds pretty interesting - I was just pondering this earlier today when running into a similar issue trying to pull apart a small model/assembly with ManipulationHandlers! Some initial thoughts on a solution are:
Might be an issue when you have more than two objects close together (i.e. how do you select the middle one when sweeping from left to right?). Would definitely help with small objects.
An extension to help selecting small objects in a cloud of other objects:
This would also help when performing two handed operations (using hand-rays) to scale or rotate small objects - otherwise it's more or less impossible to get both cursors on at the right time!
Oh and lastly; in many use cases making the bounding boxes bigger isn't a viable solution, as in the example of a mechanical assembly mesh colliders need to be used (and oftentimes convex hulls aren't appropriate either).
This is great! Apps like Fragments had a good example of using the snapping cursor behavior on the menus. It makes the far interactions much easier.
Isn't this exactly what the ICursorModifier
is for?
We appreciate your feedback and thank you for reporting this issue.
Microsoft Mixed Reality Toolkit version 2 (MRTK2) is currently in limited support. This means that Microsoft is only fixing high priority security issues. Unfortunately, this issue does not meet the necessary priority and will be closed. If you strongly feel that this issue deserves more attention, please open a new issue and explain why it is important.
Microsoft recommends that all new HoloLens 2 Unity applications use MRTK3 instead of MRTK2.
Please note that MRTK3 was released in August 2023. It features an all new architecture for developing rich mixed reality experiences and has a minimum requirement of Unity 2021.3 LTS. For more information about MRTK3, please visithttps://www.mixedrealitytoolkit.org.
Thank you for your continued support of the Mixed Reality Toolkit!
Describe the problem
Currently, when selecting objects with the far hand ray, or gaze pointer, small UI elements or focusables are difficult to target.
While it's true that making elements bigger makes them easier to target, there's a limit to the size of objects that make it possible to have presentable interfaces within the limited field of view.
Objects that are close to each other are also very difficult to select based purely on a raycast.
Describe the solution you'd like
It would be ideal to have some degree of snapping that accounts for hand jitter and lack of precision when not exactly within the bounds of a collider box.
To accomplish this, object bounds can be checked and when a cursor is near an object, the object whose bounds are closest to the cursor will get automatically selected, and stat selected even the cursor moves away from the object slightly.
This can also be done with some kind of dot product tolerance.
Describe alternatives you've considered
As I mentioned above, it is possible to make collider boxes bigger, but doing so requires a lot of manual work, and doesn't solve the issue when many objects are close to each other.
Additional context
Cursor snapping is used in the CAE VimedixAR app. I can provide a demo build to devs who require it.