Mixed Reality Capture not working on the Oculus Quest

[fabio914]

It appears that there are 2 issues that are causing the Mixed Reality support on the Oculus Quest to be broken:

• 1. The different units between the Oculus SDK and Open Brush (Open Brush uses decimeters instead of meters).
• 2. The fact that Open Brush isn't using an OVRCameraRig and it's just setting up OVRManager manually instead (here).

Issue number 2 makes the OVRExternalComposition's cameraInTrackingSpace (part of the Oculus Integration Package) to become false, making it use the incorrect worldSpacePose for the foregroundCamera and backgroundCamera cameras used for mixed reality.

I've managed to apply a workaround to the OVRExternalComposition object to fix both issues in a local copy, but that's not really ideal, since this is changing a file that's part of the Oculus Integration Package and this is currently being ignored by git.

My solution was to replace line 97 with

backgroundCameraGameObject.transform.parent = null;

replace line 123 with

foregroundCameraGameObject.transform.parent = null;

and replace lines 359:370 with

// if (cameraInTrackingSpace)
// {
    OVRPose trackingSpacePose = ComputeCameraTrackingSpacePose(extrinsics);
    trackingSpacePose.position = trackingSpacePose.position * 10; // Tilt Brush adjustment
    backgroundCamera.transform.FromOVRPose(trackingSpacePose, true);
    foregroundCamera.transform.FromOVRPose(trackingSpacePose, true);
// }
// else
// {
//  OVRPose worldSpacePose = ComputeCameraWorldSpacePose(extrinsics);
//  worldSpacePose.position = worldSpacePose.position * 10; // Tilt Brush adjustment
//  backgroundCamera.transform.FromOVRPose(worldSpacePose);
//  foregroundCamera.transform.FromOVRPose(worldSpacePose);
// }

essentially making OVRExternalComposition ignore cameraInTrackingSpace and also making it multiply the mixed reality camera's position vector by 10 (converting from meters to decimeters).

I believe we might need to adopt the OVRCameraRig on the VrSdk to be able to solve this properly, but this might cause other issues. There might be some other solution, but I'm not really an expert on Unity or the Oculus SDK.

[Scenarman]

Hi, I'm not a dev, but I need MR in my project :) What about SDK LIV ?? Is it possible to adapt it on Oculus (or the quadrants system) ? Best

[mikeskydev]

We have a PR open for LIV support (#36), we're just waiting on the submitter to move some files around. I'm not sure what the compatibility is like for Quest though as I've only used it on desktop so far.

[Scenarman]

Thank for the answer :). I have HTC Vive and Oculus Rift, so it's ok :) But I'm not shure if I have really understanding: in the last version of OpenBrush, with Steam, does LIV work? Thanks

[fabio914]

LIV doesn't work with the Oculus Quest. The Quest uses the Oculus Mixed Reality Capture mechanism that's part of the Oculus Integration Package. This is the thing that's not really working because of how the project uses the Oculus Integration Package.

[Scenarman]

Hi again, I tried LIV on Steam and my HTC, but it didn't work. I tried "Manualy" with (and after without) SDK case "on", and finaly with "Legacy" menu... I chose "OpenBrush" in the menu. What is the solution? :) Question: is OpenBrush need launch with Steam? (I think it's not possible but I ask ;)) Thanks a lot

[mikeskydev]

Tilt Brush (therefore Open Brush) has flaky support with LIV at the moment, and from experience it's usually a gamble it'll even run correctly once you get it working! You'll need to wait till #36 is merged into the main codebase and released to have proper support.

[Scenarman]

OK, thank you. If I understand it's a work in progress ;) I hope to use MR soon :) Have a good day

[mikeskydev]

Fixed in #80