Upside-down in Android platform

[iiijasdf]

Firstly thank you for such an amazing project, but I'm facing a wierd problem recently......

I'm developing with Meta Quest on OpenXR. The initial goal was to build the application into my headset and run it natively. When I develop on Standalone platform, everything looks just fine, however when I use Quest Link on Android platform, the rendered image looks upside down like this:

I tried to modify the splat renderring shader, but because I'm pretty new to shader developing, so nothing worked..... The temporal workaround I found was to switch Render Path of the camera from Forward to Deferred,and the image looks correct.

I checked the difference between Forward renderring vs Deferred rendering, I think that whether forward or deferred does not fix the problem, but since the image looks fine I'll accept it......

If anyone has found some reasonable solution please let me know, much appreciated!

[ptc-lexvandersluijs]

Hi, Thanks for sharing this issue and the resolution you found!

There is probably some 'interesting' interplay between Unity and the attached VR system, considering that the VR system can do some sophisticated things, like pre-distorting the image, judder reduction using techniques like Timewarp (https://xinreality.com/wiki/Timewarp), and so on. One hypothesis is that there is some kind of incompatibility between the Forward pipeline and Meta Quest, which doesn't manifest when using the Deferred (HDRP) pipeline. But that's just a guess.. You could try to disable some of these techniques in the control panel for your headset (if available.. on HTC Vive there are some settings, don't know about Meta) to see if that makes a difference.

In any case, if someone more knowledgeable on this topic has a better answer, I'm also very interested to hear it!

[andrewharp]

I had the same issue when building for Android/Quest 3; following the hints here I was able to get something close to working.

I edited SplatUtilities.compute and changed the cov2d computation to:

float3 cov2d = CalcCovariance2D(splat.pos, cov3d0, cov3d1, _MatrixMV, UNITY_MATRIX_P, _VecScreenParams);
cov2d.y = -cov2d.y

and centerClipPos to:

float4 centerClipPos = mul(UNITY_MATRIX_VP, float4(centerWorldPos, 1));
centerClipPos .y = -centerClipPos .y

and things seem to be rendering stereoscopically at the right pixel XYs now. However there is still what looks like some glitchy z-sorting going on that I haven't yet figured out how to squash.

[iiijasdf]

Thanks, andrewharp, your solution works pretty well! BTW I saw you developping with your Quest 3, have you tried Passthrough together with GS? I found that impressive but with performance lack when building apk with OpenXR plugin (instead of Oculus plugin)...... With the Oculus plugin, the performance of Passthrough seems nice but the GS was rendered at the different position in each eye (they were separated). I'm still trying to figure out why is this happening and I guess that the camera projection matrix was not passed properly into the GS shader with Oculus plugin but that's unsure ...... I would be so appretiated to hear if someone have some bright ideas~

[eggman28]

Did you have to do anything to make it run in Quest 3 ? It runs on Windows for me but is quite laggy when streaming via Virtual Desktop (I don't see option to disable Smoothing on Steam VR) even when running as built Windows app, not via Unity. When built as Quest 3 APK, I only seem to see black screen with 3 dots and I can't seem to exit the app and need to restart the headset.

[iiijasdf]

Quite strange... I'm using Quest Pro and can see my splats perfectly but with a low framerate, guess the reason is the combination of Meta Passthrough and the original OpenXR feature package. Have you checked your project settings? It may need IL2CPP, .net framework, arm64, Android 10.0 least, multi-pass, depth 16bit op higher to make it work in Quest devices.