Rendering glitches

[PowZone]

I'm working with an architectural model (20M+ triangles) using azure remote rendering on Unity. In editor everything is fine and the remote model is perfect but on Hololens 2 there are several rendering glitches.

I tried to reproduce them, they are only visibile on Hololens: https://imgur.com/a/b74wzUK

Two cases:

• Even standing still some pieces of texture move without reason
• There is a wavey effect on some mid-small elements

And often it glitch on world position when I walk around the model. I don't think it is a bandwidth issue because I have a gigabit connection.

Unity 2020.3.13f1 (but it's the same with 2020.3.25f1) Azure Remote Rendering 1.0.39 MRTK 2.7.3 OpenXR 1.2.1

[ChristopherManthei]

Hi @PowZone, thanks for reaching out to us. What you are describing sounds like depth late stage reprojection artifacts. As time passes between the image being rendered by the server and it being presented on your device the predicted pose can by slightly off. To stabilize the image this is corrected by the hardware via DLSR which reprojects the rendered image according to the depth buffer. This can cause warping at depth discontinuites. The effect scales with your latency to the server, so can you post your round trip latency as seen in the quickstart status panel or the equivalent of the showcase app?

To make sure this is indeed the problem you are seeing can you please try the following:

1. Enable local pose mode. This is already the case in the latest Quickstart and Showcase example apps. This will remove warping between local and remote content, but won't help you with warping of the remote content against itself. However, it will allow you to capture the reprojection artifacts in a video capture.
2. Create an MRC capture of your problem and post it here. You can start the video capture either via the button at the bottom of the start menu or via the device portal as described in the link above.

[WikkidEdd]

One other aspect worth looking out for is that transparent materials don't by default write to the depth buffer. So anything with a transparent material on will be unstable/wavey (due to what @ChristopherManthei says about DLSR). Looking at your screenshots this could be the case, the glass building, trees, and railings could all be transparent materials.

[PowZone]

Hi @ChristopherManthei, thank you! Enabling local pose mode makes the model correctly fixed in the space (like a local 3d object).

I tried a MRC Capture but when I enable capture, or just a screen mirror on desktop, the remote rendering is completely glitched and flashing between invisible and an heavly distorted image. On hololens the distorsion is on right eye, on left eye it seems ok. In the recording it records only the distorted rendering.

@WikkidEdd we removed grass and trees from the top of the building, they had a lot of transparencies, but small elements like external stairs continue to have a wavey glitch even if they are all polygons. We are checking model and textures for other transparencies.

Do you know if these glitches about transparencies will be solved in next releases or it's just better to avoid them completely?

[ChristopherManthei]

Hi @PowZone , we fixed the Mixed Reality Capture glitch in version 1.0.34. Which ARR version are you using and which graphics binding (OpenXR or WMR)?

Transparencies can't be reprojected unless you enable TransparencyWritesDepth. Doing so is fine for fairly opague objects but will obviously distort the objects behind the transparent object as there is only one depth value per pixel. It's a tradeof that you can do on a case-by-case basis.

For foilage like grass or tree leaves it might be best to use alphaClipEnabled instead of using transparency.

As for the stairs, I would need to see a video to be sure but you could also be running into floating point precision issues. If you look at the *.info.json file generated during conversion in your storage account you can have a look at the outputStatistics.boundingBox extents. If these are very large it would be best to enable recenterToOrigin during conversion.