search

mrdoob / three.js

JavaScript 3D Library.
https://threejs.org/
MIT License
101.78k stars 35.31k forks source link

three.js VR renderer does frustum culling based on only the left eye, causing premature clipping on the right eye. #13236

Closed ngokevin closed 5 years ago

ngokevin commented 6 years ago
Description of the problem

The VR renderer does frustum culling based on the frustum of the left eye camera. It applies the same culling to the right eye. As a result, when an object leaves the left eye's view, the user can see the object flash out on the corner of the right eye periphery.

Saw https://github.com/w3c/webvr/issues/203 was linked.

https://github.com/mrdoob/three.js/pull/11523

Three.js version
Browser
OS
Hardware Requirements (graphics card, VR Device, ...)

VR Device

ngokevin commented 6 years ago

@mrdoob From talking yesterday, yeah, I can notice in VR from the Rift in my right eye as objects leave the left eye's frustum.

mrdoob commented 6 years ago

Is that with https://threejs.org/examples/webvr_cubes.html?

ngokevin commented 6 years ago

Yeah, I can notice it there and on our own projects with Rift. Should be more apparent if you have longer objects.

mrdoob commented 6 years ago

Longer vertically I guess?

amakaseev commented 6 years ago

you can see this problem in a code

_projScreenMatrix.multiplyMatrices( camera.projectionMatrix, camera.matrixWorldInverse );
_frustum.setFromMatrix( _projScreenMatrix );

but

// HACK @mrdoob
// https://github.com/w3c/webvr/issues/203

cameraVR.projectionMatrix.copy( cameraL.projectionMatrix );
amakaseev commented 6 years ago

I think increase fov in cameraVR.projectionMatrix, will solve this problem

mkeblx commented 6 years ago

Just changing FOV not what wanted. The linked PR at top looked good conceptually to work in the general case. Would that not work?

amakaseev commented 6 years ago

Yes there almost everything is truly described except the picture in the first post Correct frustum has to look so: frustum

ngokevin commented 6 years ago

The culling issue has seemed to worsen on landing of r90?

Try https://threejs.org/examples/webvr_vive_paint.html in a Vive, all objects disappear including the controlller unless they are the far left side of field of view.

mrdoob commented 6 years ago

The culling issue has seemed to worsen on landing of r90?

Did it worsen for all headsets or just for Vive?

Try https://threejs.org/examples/webvr_vive_paint.html in a Vive, all objects disappear including the controlller unless they are the far left side of field of view.

I don't have a Vive. @dmarcos do you mind testing this? I have the feeling #13158 has something to do with this.

dmarcos commented 6 years ago

I can see the regression and narrowed down the changes that caused it: https://github.com/dmarcos/three.js/commit/7bf42e19134f26e99bf555d19a934b9f2ff973c8

Looking closer to understand what happened. @brianpeiris ?

I opened a PR in case you want to get it fixed for now (https://github.com/mrdoob/three.js/pull/13350). I tested on Vive / Oculus.

brianpeiris commented 6 years ago

Not sure what's going on here. I'll have to get my hands on a Vive to understand it. I'll add comments to #13350 as well.

brianpeiris commented 6 years ago

I haven't been able to reproduce any culling issues in my Vive with r90 (d55897b). I see a culling issue in my Rift with r90 and r89 as well, so I don't think this is the standing matrix change after all.

brianpeiris commented 6 years ago

Correction: dmarcos helped me repro it. It's very intermittent for me. But when it does occur it shows an obvious culling issue on the left of the frustum, where it culls too early.

dmarcos came up with these steps to repro:

  1. Exit SteamVR and Firefox
  2. Start Firefox
  3. Visit https://threejs.org/examples/webvr_vive_paint.html
  4. Enter VR
  5. Exit Firefox
  6. Visit https://threejs.org/examples/webvr_vive_paint.html
brianpeiris commented 6 years ago

Alright, I think I've solved this correctly in #13414. I was able to reproduce it much more consistently if I setup my Vive playspace so that it was rotated 90 degrees (or any angle other than near-zero) from my screen.

LAGENCECREE commented 6 years ago

I confirm https://threejs.org/examples/webvr_cubes.html have a camera culling issue on window 10/vive/last firefox , far distance is extremely wrong on both eyes + left eye controlling the culling ( gfx card, Radeon rx 480 )

dmarcos commented 6 years ago

https://github.com/mrdoob/three.js/pull/13414 solves part of the problem. Frustum culling is still performed with just one eye resulting in disappearing objects. @mrdoob should we reopen or you prefer to have a discussion somewhere else?

brianpeiris commented 6 years ago

Thanks for the clarification @dmarcos. I didn't realize there was still a culling issue after my fix because it's not very obvious in the "cubes" demo. It's visible, but not when you're in the HMD -- you can see it more clearly in the browser's stereo view though.

@omgitsraven provided a clear demo in aframevr/aframe#3488 which I've converted to three.js code here: https://edge-disappear.glitch.me/ https://glitch.com/edit/#!/edge-disappear

ngokevin commented 6 years ago

This still happens (e.g., on Oculus Rift). Objects disappear out of the corner of the right eye while turning head. Is there an issue this is being tracked?

mrdoob commented 6 years ago

Oh, I guess this issue got closed automatically...