search

microsoft / MixedRealityToolkit-Unity

This repository is for the legacy Mixed Reality Toolkit (MRTK) v2. For the latest version of the MRTK please visit https://github.com/MixedRealityToolkit/MixedRealityToolkit-Unity
https://aka.ms/mrtkdocs
MIT License
6k stars 2.12k forks source link

Improved Mixed Reality Capture #7567

Closed Naphier closed 4 years ago

Naphier commented 4 years ago

Describe the problem

When using Mixed Reality Capture (MRC) holograms become unstable and hand-tracking is offset. The hand tracking offset can be overcome with a setting, but even without worrying about the hand tracking the captures are twitchy and unstable. This means that it is very difficult to produce video demos of applications to showcase the HoloLens apps.

Describe the solution you'd like

A more stable method for MRC or at the very least tools or a clear workflow on capturing good demo videos. MS's stage videos are excellent and for this product to be a success developers and producers also need a tool like this to market their products.

If this is something being worked on that's great. Please provide a timeline. In the meantime it would be excellent to have a guide for how to get the best quality demo video as is possible.

Thanks!

keveleigh commented 4 years ago

Yep, this is a current known issue with third camera render (MRC hologram alignment) and Unity. I believe most of the fixes regarding instability should be in the latest version of 2019.3 and the next version of 2018.4, with a few more fixes still in progress.

I think some of the instability in MRC holograms without hologram alignment was due to an OS bug. It should be fixed, but I'm not sure if it's made it out into public OS version yet.

Naphier commented 4 years ago

Very good to hear! Thanks for the hard work! Hoping to see this closed out soon. Will see how it is in the newest 2019.3. Had some bad physics issues there (report incoming) so regressed back.

Naphier commented 4 years ago

I'm happy to report that my lead engineer says the recording as of Unity 2019.3.7 is "perfect"! There's still some clipping issues, but we can work with that at least. Thanks for the hard work!

keveleigh commented 4 years ago

Glad to hear it! I'll keep the clipping issues tracked with Unity

Naphier commented 4 years ago

Do you have an issue tracker link for that so I can follow / upvote?

keveleigh commented 4 years ago

@Naphier I don't at the moment. I'll let you know if we get one! I'm tracking it directly with them.

camnewnham commented 4 years ago

I think some of the instability in MRC holograms without hologram alignment was due to an OS bug. It should be fixed, but I'm not sure if it's made it out into public OS version yet.

We are experiencing this issue with Unity 2018.4.19f1 when the "Render from PV camera" option is checked in MRC. It is resolved issue when using OS insider builds. I'd hoped these would have made their way into public release but as of the August Update (2004) it is still an issue.

To clarify, the issue in this case is that the holograms in the recording are extremely jittery, even though they remain stable for the user. As this likely lies out of MRTK scope (fixed in pre-release OS), is there any timeline for insider builds for fixes like this?