How to make it work with Meta XR OVR Interaction for grabbing using hand and ray ?

[ramkeshcse084]

I want to add feature of Grabbing . For this i used OVR camera rig and from example scene. When i test in Meta device my meta movement character hands is not sync with my Interaction hands and controller. There is little difference in it. We will use this in multiplayer .What we have to do for it ? I am sharing video for reference.

https://github.com/user-attachments/assets/2e513fa5-4d42-48df-847c-4da1c607ac64

Please guide me what will be best way to do it ? Thank You

[sohailshafiiWk]

Hello, can you take a look at the ISDK integration sample scene? You may find documentation here https://developer.oculus.com/documentation/unity/move-samples/#isdk-integration about that sample.

[ramkeshcse084]

Hello, Thank you for quick reply. I just tested Sample MovementISDKInteraction scene .

https://github.com/user-attachments/assets/b0e332fe-0c9b-41d9-aa93-11f7b05886ee

Grabbing is not working there. When I just opened scene Hands rotation was not ok, when I start recording rotation of hands come in good position. Here is reference video :

[ramkeshcse084]

I followed docs https://developer.oculus.com/documentation/unity/move-samples/#isdk-integration to integrate ISDK on my character. Grabbing is working but same issue with additional one more issue. 1. Gap in Character hands and synthetic hands 2. Character wrist is getting disconnected to character. I am sharing Screenshot and video for better understanding.

https://github.com/user-attachments/assets/612bcb30-d6c9-465f-8bb9-a9d0d42980f7

[sohailshafiiWk]

Can you tell us the versions of the Meta XR packages that you've included? You can find this in Unity Package Manager or your project's manifest.json. Please list the versions of packages with the prefix com.meta.xr.sdk, such as com.meta.xr.sdk.core and so on. I assume that you are using Unity 2022.3.39f1 as well?

[ramkeshcse084]

Yes, Unity version is same as mentioned. Here is Meta SDKs with version from package manager:

Thank You

[sohailshafiiWk]

Hello, please download the attached project. Go to the Meta Movement package in the Unity Package Manager, and import the advanced samples. Please try the ISDK scene as it is imported into the Assets folder. Testv67Int.zip

Please note that you might see an error related to Unity input system. We are currently looking into that issue.

[ramkeshcse084]

Hello, Thank You. I just tested ISDK scene in given project, having same issue not able to grab blue cup. Here is video for ref.

https://github.com/user-attachments/assets/daf477b7-176d-44e1-9e00-907091a5c132

[sohailshafiiWk]

Could you tell us what OS version your device is running on? This can be found in the "Software update" section of system settings. Which device is this running on (Quest 2? Quest 3?)?

[ramkeshcse084]

Here is version details :
Thanki You

[sohailshafiiWk]

Thank you for the information. We have been looking into this problem and will report back when we have a fix available.

[ramkeshcse084]

ok. Thank You. We are waiting for this. When fixed done, please us know.

[sohailshafiiWk]

Hello. We have provided this fix (https://github.com/oculus-samples/Unity-Movement/commit/d837b0ccd72a9b9e31115d656548172bb08b1ad0) to #dev branch and will do a release to main and do a document update once that process is complete.

In the meantime, please make the following changes that the fix applies, which use Capsense (https://developers.meta.com/horizon/documentation/unity/unity-capsense/):

1. Delete OVRControllersHands and also the references to its SkeletalHandsAdjustment component under ArmatureSkinningUpdateRetargetSkeletonProcessor; this means deleting the entries LeftControllerHandSynthentic and RightControllerHandSynthentic. The LeftHand/RightHandSynthetic references will remain.
2. In OVRCameraRig->OVRManager, navigate to the "Controller Driven Hand Poses Type" and set it to Natural.
3. In each OVRHandPrefab under the related hand anchors (such as OVRCameraRig->TrackingSpace->LeftHandAnchor->OVRHandPrefab), navigate to the OVRHand component and set "Show State" to Always (not something like "Contoroller Not In Hand").

Because OVRControllerHands will be deleted and you can rely on OVRHands to be driven by controllers inputs, the scene will be simpler. The ISDK scene that is fixed in that revision on the #dev branch can be used as a reference.

[ramkeshcse084]

ok. Please let me know when new version of Meta movement SDK will arrive. I have another question related to Meta movement SDK update; I will ask in other thread.

[ramkeshcse084]

I

https://github.com/user-attachments/assets/b4c3ceeb-5e22-41cd-96cf-957e31f90968

followed given steps, now ISDK Samples scene started working. When I implemented this step with my locomotion character, grabbing is working with hand tracking but there is gap in Character hands and tracking hands. Any idea how to fix it? We are at initial stage of issue . Here is video for ref.

[sohailshafiiWk]

For this character, navigate to the RetargetingLayer's RetargetingProcessors section, go to the "Correct Hand" processor and disable "Use Secondary Bone Id" checkbox for the "Left Hand Processor" and "Right Hand Processor" sections.

With this setting off, the hand should look more accurate, however the upper arm won't be rotated toward the tracked lower arm position. This means that the arm bones above the wrist might look incorrect.

Furthermore, the mesh appears to disconnect from itself when the bone positions are stretched beyond what the character is designed for. You can consult https://developers.meta.com/horizon/staging/moveUnity/documentation/unity/move-body-tracking#modifying-character-height-to-match-varying-user-heights for more information regarding how to adjust the model.