search

servo / webxr

Bindings for WebXR
Mozilla Public License 2.0
81 stars 23 forks source link

Image is upside-down when rendering with Oculus OpenXR runtime #216

Open msub2 opened 5 months ago

msub2 commented 5 months ago

While testing Servo on my Win10 machine I wanted to track down the cause of the in-headset image being flipped despite my earlier patch updating the projection matrix logic. What I found was that SteamVR via Virtual Desktop displayed the image rightside-up, but with the Oculus runtime it renders upside down. I'm not sure what the specific cause of this is yet.

msub2 commented 5 months ago

Only other instance of a similar-sounding issue I could find, unfortunately the XR_FB_composition_layer_image_layout extension is only available on headsets, not the PC Link runtime.

jdm commented 5 months ago

https://github.com/servo/webxr/blob/main/webxr/openxr/mod.rs#L635 might be related. Not sure how to determine if it's needed for a particular openxr implementation though.

msub2 commented 5 months ago

Oh wow I didn't even see that, that definitely seems like the most likely culprit though. We can get runtime name once the XrInstance is created, so it should just be a matter of keeping an internal flag for it. I'll test and make sure that's the actual fix first though.

msub2 commented 5 months ago

Hmm, commenting those two lines out didn't seem to have any effect. I'll keep digging, maybe see what the actual FOV values are that we're getting

msub2 commented 5 months ago

Update: Sent a message in the Khronos Discord and got relevant info image This would explain why commenting out those lines did nothing, since they were never actually swapping in the first place

msub2 commented 5 months ago

Matthieu suggested inverting the projection matrix being calculated in fov_to_projection_matrix, so I'll try that

msub2 commented 5 months ago

So by swapping fov up/down in fov_to_projection_matrix instead we can fix the texture coord mismatch, but it seems like now there's a culling issue (planets look odd and the spherical skybox is no longer visible). I'm unsure what the best way would be to remedy that here, I think I can dip into the d3d11 device but that requires some unsafe code I think and I'm not super familiar with the APIs.

msub2 commented 5 months ago

@jdm Do you know if there's a way to conditionally adjust face culling for the rendered image? I tried dipping into the d3d11 device and adjusting rasterizer state, adding a pixel shader, but to no effect (I assume because most stuff is happening in surfman and modifying the device doesn't affect that).

jdm commented 5 months ago

I do not. The extent of my d3d experience is the code in this repository.