[Feature] Allow the MixedRealityPlayspace to be a Moving Platform

[Waffle1434]

For the current project I'm prototyping for a company, they want the HoloLens perspective as if it was inside a moving vehicle, such as being inside a virtual flying helicopter. This is possible in MRTK, but WLT seems to only understand locking the camera to the world, not locking the camera to a Transform's local space.

I essentially want to parent the AdjustmentNode (Orienter), which includes MixedRealityPlayspace, & Camera, to a GameObject that may move at any time. Currently the only solution is to move the entire world and keep the WLT & MixedRealityPlayspace stationary. This is a very poor solution though, its high maintenance, it doesn't allow me to use static batching for the world, and would have to alter the gravity vector if the vehicle pitches or rolls.

[pjaydev]

I'm not sure if this exactly fits your demands, but have you tried out the AlignSubtree feature, which was introduced with WLT v1.5.0 ?

This allows to let WLT adjusting the global pose of one parent GameObject of your choice instead of moving the whole playspace/camera.

To use this feature, you'd need to uncheck Apply Adjustment in your WorldLockingContext and add an AlignSubtree component to the GameObject, which will be the parent of all GameObjects which you wish to be adjusted by WLT.

There is some documentation in the reference: https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/DocGen/Temp/api/Microsoft.MixedReality.WorldLocking.Core.AlignSubtree.html?q=alignsubtree

[fast-slow-still]

@Waffle1434 , what happens when you parent the AdjustmentNode to a moving GameObject (transform)?

[Waffle1434]

@Waffle1434 , what happens when you parent the AdjustmentNode to a moving GameObject (transform)?

When parented, the functionality is unchanged. Moving the parent just results in the AdjustmentNode Orienter cancelling out the movement such that the camera remains in the exact same place in world space.

I believe AdjustmentNode should probably be getting & setting its position via localPosition, instead of position (if it is) which ignores all transform heirarchy. If AdjustmentNode has no parent, localPosition is identical to position, if it does have a parent then it'll be offset by it.

[Waffle1434]

I'm not sure if this exactly fits your demands, but have you tried out the AlignSubtree feature, which was introduced with WLT v1.5.0 ?

This allows to let WLT adjusting the global pose of one parent GameObject of your choice instead of moving the whole playspace/camera.

To use this feature, you'd need to uncheck Apply Adjustment in your WorldLockingContext and add an AlignSubtree component to the GameObject, which will be the parent of all GameObjects which you wish to be adjusted by WLT.

There is some documentation in the reference: https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/DocGen/Temp/api/Microsoft.MixedReality.WorldLocking.Core.AlignSubtree.html?q=alignsubtree

This is still "world locked" though. I need transform-space locked.

[fast-slow-still]

@Waffle1434 , could you clarify what you've tried, what you hoped would happen, and what actually did happen? Please make no assumptions about what should be obvious. Also, try to simplify as much as possible, focus on describing what is happening and what you would like to happen, and in concrete terms. You might find it helpful to create a very simple Unity project showing what you believe to be the current incorrect behavior.

[Waffle1434]

@Waffle1434 , could you clarify what you've tried, what you hoped would happen, and what actually did happen? Please make no assumptions about what should be obvious. Also, try to simplify as much as possible, focus on describing what is happening and what you would like to happen, and in concrete terms. You might find it helpful to create a very simple Unity project showing what you believe to be the current incorrect behavior.

I'm mocking this up on my home machine so its not going to be exact. Lets say I have this scene for the HoloLens 2. World objects are static, the AdjustmentNode & MixedRealityPlayspace are at the origin and work fine:

I want to parent the AdjustmentNode to Platform, such that moving the platform moves the camera rig as if you were standing on the platform. The platform could be thought of as a vehicle, such as a helicopter. This is to make it such that if you are standing in a real room with the HoloLens on, the virtual helicopter's interior floor would feel stationary, and the world would appear to move around you as you fly: If I had a real-world mockup of a helicopter cockpit, I would use WLT's space pins to keep the virtual cockpit aligned with the real. This is the reason for using WLT in this context.

This is what happens if I try to do that with WLT. The camera rig remains at the same world positon & rotation, instead of moving with the platform, awkwardly leaving the camera behind as the platform moves around the world. It is as if the parenting hierarchy above AdjustmentNode was ignored:

WLT is counter-transforming AdjustmentNode to keep the camera rig scene locked, instead of keeping it parent locked:

For certain applications you need the AR playspace to be a moving platform. We should be able to lock the camera rig to a real world stationary vehicle cockpit/interior, for something such as a flight simulator. This is how all vehicular VR games work.

[fast-slow-still]

Great description. I'll try to figure out what's going on. Many thanks!

[fast-slow-still]

Okay, I tried what I think you are requesting, and I think it worked as you want.

PlatformTrim.zip

I've trimmed this down to the bare essentials to avoid a hugely redundant download. You should be able to just drop these assets into your working project, and then export the scene.

If you look at the scene 1_AddWLT, it has a script on the Platform node which moves it in a circle around the origin (set for 6 meter radius), while always looking at the origin. That's your vehicle.

Ajustment node etc down to camera are attached to the Platform node.

After build and deploy, it looks like the stationary cubes are flying in a circle, but actually it's the camera and attendant spheres that are orbiting the origin. So spheres are the inside of your vehicle moving with the camera, the cubes are the stationary exterior.

At one point, I had accidentally set the Platform node to be the Adjustment Frame in the WLT Manager. As documented, nobody but WLT can modify the Adjustment Frame transform. With it incorrectly setup like that, I saw the behavior you are reporting, with the CircleCam script's modifications to the Adjustment Frame immediately overwritten. But setting the Adjustment Frame correctly back to the Adjust node fixed it up.

So, try the attached sample and see if it works for you. If it does, compare it with your setup, paying especial attention to the Adjustment Frame not being the target of any transform writes. Your transform modifications should be to the parent of the Adjustment Frame.

If the sample doesn't work for you, let me know and we'll go from there. So if you can let me know either way, that would be great. Or also let me know if I completely misunderstood you.

Thanks again for the detailed writeup!

[fast-slow-still]

[fast-slow-still]

For using SpacePins with a moving platform, I first had to make some sense out of what it means to pin something to the physical world when its coordinates are constantly changing. I think I managed something self-consistent and useful.

See the new Vehicle sample in release 1.5.8 (PR #267 ).

I'm updating the documentation now, and will post a link here when it is live.

[fast-slow-still]

Here's a link to the documentation on the sample:

https://docs.microsoft.com/en-us/mixed-reality/world-locking-tools/documentation/howtos/samples/vehicle

[Waffle1434]

Thank you for everything, I haven't quite been able to try it out as I've been busy but I'll be sure to

[fast-slow-still]

Thank you again for the feedback, it was a good scenario to support.

[Waffle1434]

I was able to test Vehicle.unity, its working quite well!

The only potential issue I'm seeing right now is while manipulating the space pin, the pin's rotation becomes scene stationary, meaning trying to align something results in it spinning until you let the pin go. I figure this may be a MRTK-ism as they don't really anticipate people manipulating objects on moving platforms.