Mirror stereoscopy is offset differently on each eye for WMR/Pimax headsets

[shiftyscales]

The stereoscopic texture used on mirrors within Neos is currently incorrectly configured/displayed. The further away a player is from a mirror, the more the images separate, resulting in double vision.

The left eye/image is on the bottom, and the right eye/image is on the top. This can be visually disorientating to look at.

[Frooxius]

What makes you think they're separated bottom/top? Mirrors don't really work that way, it's hardcoded to use double wide stereo texture, so it can't be bottom/top. Whatever you're seeing is likely something else.

I haven't noticed any incorrect rendering on mirrors myself. I know certain headsets have issues (like Pimax), likely because they do some strange modifications to rendering matrix. Which headset are you using?

[shiftyscales]

I am using the Samsung Odyssey+ via SteamVR. And I think they're separated that way because I can visually see that they are. I also got confirmation of this from Artleck in his OOT project world (not sure what headset he's using- I believe Index though.)

The easiest way to demonstrate that is to have a line of objects you can see in the mirror, and to back away from the mirror, you will see them slowly separate out top, and bottom. The left eye is directly below the right eye, @Frooxius.

[Frooxius]

That doesn't really make much sense to me. The textures are hardcoded to be left/right, there's just no way for them to be bottom/top. Can you send me screenshot of what you're seeing in the inspector when you look at the texture?

I don't experience the same rendering issue on my end, the mirrors work perfectly. I have an Oddysey+ lying around though so I could check with that one. Anyone with Index experience visual artifacts with mirrors?

[ProbablePrime]

No issues here, Only when combined with a fog volume and I know the OOT world has one so it could be that.

[Frooxius]

Yeah certain graphical features aren't unfortunately compatible with mirrors, like the fog volume or any shaders with grab pass (e.g. filters, blur, refraction materials).

Although that has nothing to do with the left/right and top/bottom separation of the render texture.

[shiftyscales]

I'm in a group with Artleck, and Vigilabo. Vigilabo is on Oculus/not using SteamVR, and the claims mirror renders properly. Artleck is using a Vive, and is experiencing the same issue I am on WMR.

So far it seems to be pretty much everything rendered in the world including objects, avatars, etc. that are having this issue. An issue that seemingly is somehow related to SteamVR/OpenVR specifically.

If you're available, I could invite you to his session, @Frooxius?

[Frooxius]

Does it happen specifically in that world? What if you open another world and spawn a mirror there? Do you experience the same issue?

[shiftyscales]

Even using my home/local world, and spawning in a "local mirror" I get the same effect on my avatar (the stock/default Renamon).

[cristian2452]

i can confirm this using hp wmr. the rendering is offset the more far you are from the mirror make it a doubble vision effect. also happen with all types of mirrors

[Anomalous]

I've had no problems with mirrors on the Index headset, but they're definitely broken on the Quest. I didn't bother reporting it before because the Quest build is still a WIP, but for the sake of completeness I figured I'd mention it here. The offset there is side to side, just way too much.

[Frooxius]

I'll try doing some testing on the Samsung Odyssey+ and I'll see if I can replicate the behavior. Those headsets had issues in the past with them too, because their projection matrices are kinda weird (and sometimes even change significantly with some updates).

Quest is definitely WIP, I haven't looked there much into mirrors yet.

Thanks for all the info!

[MR-Alex42]

I want to add that this issue also effects Pimax devices (tested with 5K+ and 5K-XR).

[KacperLa]

Just an update for Pimax headsets, turning on the "Compatible with Parallel Projection" option in Pi tool will fix the mirror issue that was mentioned here and that I showed pictures of in issue #548 .

[shiftyscales]

Still have no idea why this rendering issue exists, but it's definitely still present on Windows Mixed Reality headsets. Easiest to see by going to the top border of a mirror at a distance, and winking, noting that the right eye renders a view point considerably higher than the left in the mirror.

Probably an issue either on Unity or SteamVR's end- but I don't think I could adequately diagnose which.

[PolygonBird]

So.. I know the bugs with Pimax HMD's having issues with the Portal/mirror shader. The problems is not a new topic but I thought I'd chip in being one of those few niche users who has this HMD. (I use a Pimax 5K XR)

I don't see any pictures here that better visualizes the problem that is being explained, kinda impossible to explain because it is rather disorientating to look at.. So here is one I took. My avatar should be more in the center or towards the middle between each eye of the HMD but Neos does the exact opposite

Anyway, I thought I should mention since I have been playing around with my own projects and ask if any of the devs here tried out the SDK Pimax provide? https://www.pimax.com/pages/developer I gave it a go in a Unity scene of mine and found it to be literally drag and drop. Though I have no idea how you implement your cameras and how the mirrors are projected in Neos if it would make any difference.

It is uncommon to face issues in other games with a Pimax HMD though from experience 3 out of 10 games have odd problems with the Pimax, pretty much all of them that have issues has always been the culling being too narrow. Seeing objects pop at the sides of the screen.. This mirror issue is truly a unique issue to Neos.

The only solution for anyone to play Neos without headaches is to run it in "Compatible with parallel projections" mode, but that has a HUGE performance and visual sharpness cost. Depending on settings you choose.

You can learn more about parallel projections here~ https://community.openmr.ai/t/closer-look-at-pimax-parallel-projection/20510

[Frooxius]

Thanks for the info, unfortunately their SDK doesn't help there much, as we need to apply the fix to our own implementation of mirrors/portals, which is a combination of a rendering code and a shader. Without the hardware to test on, this is a bit difficult as we can't test where exactly are things going wrong and iterate on a solution.

If there was a simulator at least which would reproduce the problem that could help. We'd have to see if we can borrow the hardware for testing somewhere otherwise.

[PolygonBird]

So far I have determined this issue with Portals and mirrors has a lot to do with Pimax angling their displays 10° compared to other HMD's that have the displays flat. (This being what parallel projection corrects for a 'fix' with a huge visual compromise)

(I'd be willing to donate NCR for one of the devs to purchase any Pimax HMD that has the tilted displays. This bug really causes me and very likely other users great discomfort. Especially in places I cannot turn it off. Its really been putting me off personally. I really do enjoy Neos, but personally really breaking the experience for me.)

[YaBoi3K]

Thanks for the info, unfortunately their SDK doesn't help there much, as we need to apply the fix to our own implementation of mirrors/portals, which is a combination of a rendering code and a shader. Without the hardware to test on, this is a bit difficult as we can't test where exactly are things going wrong and iterate on a solution.

If there was a simulator at least which would reproduce the problem that could help. We'd have to see if we can borrow the hardware for testing somewhere otherwise.

If you have access to a valve index, you can use that for non-parallel displays. The index actually uses non-prarllel displays, but the angle is shallow enough (5 degrees) that they just leave parallel projections on all the time without too much of a performance hit. Pimax headsets are between 10-15 degrees, which ends up at about a 30% performance hit. Note that the angle does not scale linearly to the performance hit and follows more of an exponential curve.

To disable parallel projections in SteamVR, add and/or set the following line: "steamvr" : { "renderCameraMode" : "raw" } To re-enable parallel projections, set it to: "steamvr" : { "renderCameraMode" : "parallel" }

[PolygonBird]

I sadly do not have an Index. It's very hard to obtain one in Australia. Pimax does have Paralel projection to enable. However it may look okay on the Index, it can look really awful. Basically looks incredibly low res or better worded 'blurry', text is almost unreadable unless you get really close. You basically have to choose between a clear image wide FOV broken portals, or Low FOV poor image fixed mirrors. You have to go to absurd resolutions just to get something that 'looks' okay... This post here is a good reference to help visualize the resolutions. https://community.openmr.ai/t/closer-look-at-pimax-parallel-projection/20510

Recommended render target size (SteamVR SS=100%, PiTool RQ=1.0): PP off: [4268, 2632] PP on: [6948, 3288]

------- Original Message ------- On Sunday, February 27th, 2022 at 09:16, 3Ktone @.***> wrote:

Thanks for the info, unfortunately their SDK doesn't help there much, as we need to apply the fix to our own implementation of mirrors/portals, which is a combination of a rendering code and a shader. Without the hardware to test on, this is a bit difficult as we can't test where exactly are things going wrong and iterate on a solution.

If there was a simulator at least which would reproduce the problem that could help. We'd have to see if we can borrow the hardware for testing somewhere otherwise.

If you have access to a valve index, you can use that for non-parallel displays. The index actually uses non-prarllel displays, but the angle is shallow enough (5 degrees) that they just leave parallel projections on all the time without too much of a performance hit. Pimax headsets are at about 15 degrees, which ends up at about a 30% performance hit. Note that the angle does not scale linearly to the performance hit and follows more of an exponential curve.

To disable parallel projections in SteamVR, add and/or set the following line: "steamvr" : { "renderCameraMode" : "raw" } To re-enable parallel projections, set it to: "steamvr" : { "renderCameraMode" : "parallel" }

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you commented.Message ID: @.***>

[aecii]

Adding this problem is also present on Pimax 8kx. Double vision on all mirrors in Neos Essentials / mirrors inventory folder (and some water effects in certain worlds) Dash mirror works fine though.

~ wauf!

[shiftyscales]

@aecii - as observed by another user above:

Just an update for Pimax headsets, turning on the "Compatible with Parallel Projection" option in Pi tool will fix the mirror issue that was mentioned here and that I showed pictures of in issue https://github.com/Neos-Metaverse/NeosPublic/issues/548 .

[aecii]

Thanks. Seems that was already on by default. I will try toggling it on/off to see if i get any results. EDIT: Toggling off and back on fixed it. >_<