mrdoob / three.js

JavaScript 3D Library.
https://threejs.org/
MIT License
102.47k stars 35.36k forks source link

WebGLRenderTarget has some problems in VR mode #15614

Closed zhudongwork closed 4 years ago

zhudongwork commented 5 years ago
Description of the problem

I am trying to render a scene into another scene using WebGLRenderTarget in VR mode. I created two scenes - mainScene and bufferScene. I draw a red cube in bufferScene, then render the bufferScene to the texture bufferTexture using WebGLRenderTarget . bufferTexture as the texture of mainScene. When I enter VR mode and turn the head to observe the entire scene, mainScene is fine, but the cube in bufferScene is rendered by split screen, and it will follow my head movement. Generally speaking, in the vr scene, the object should not follow the head movement. What caused this situation? I have put the code in codeopen.

var mainScene = new THREE.Scene();
mainScene.position.set(0,0,0);
var width = window.innerWidth;
var height = window.innerHeight;
var camera = new THREE.PerspectiveCamera( 70, window.innerWidth / window.innerHeight, 1, 1000 );
camera.position.z = 10;

var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.vr.enabled = true;
document.body.appendChild( renderer.domElement );
document.body.appendChild( WEBVR.createButton( renderer ) );

//This is where we create our off-screen render target
//Create a different scene to hold our buffer objects
var bufferScene = new THREE.Scene();
//Create the texture that will store our result
var bufferTexture = new THREE.WebGLRenderTarget( window.innerWidth, window.innerHeight,
                                { minFilter: THREE.LinearFilter,
                                    magFilter: THREE.NearestFilter});

//Let's create a red box
var redMaterial = new THREE.MeshBasicMaterial({color:0xF06565});
var boxGeometry = new THREE.BoxGeometry( 5, 5, 5 );
var boxObject = new THREE.Mesh( boxGeometry, redMaterial );
boxObject.position.z = -10;
bufferScene.add(boxObject);//We add it to the bufferScene instead of the normal scene!

////////////////////////////Now we use our bufferTexture as a material to render it onto our main scene
var boxMaterial = new THREE.MeshBasicMaterial({map:bufferTexture.texture});
var boxGeometry2 = new THREE.BoxGeometry( 5, 5, 5 );
var mainBoxObject = new THREE.Mesh(boxGeometry2,boxMaterial);
mainBoxObject.position.set(0,0,-10);
mainScene.add(mainBoxObject);

///And a blue plane behind it
var blueMaterial = new THREE.MeshBasicMaterial({color:0x7074FF})
var plane = new THREE.PlaneBufferGeometry( window.innerWidth, window.innerHeight );
var planeObject = new THREE.Mesh(plane,blueMaterial);
planeObject.position.z = -15;
mainScene.add(planeObject);

//Render everything!
function render() {
  renderer.setAnimationLoop( render );
  renderer.render(bufferScene,camera,bufferTexture,true);
  renderer.render( mainScene, camera );

}
render();
Three.js version
Browser
OS
Hardware Requirements (graphics card, VR Device, ...)
Mugen87 commented 5 years ago

Possible duplicate of #15393

jsantell commented 5 years ago

What browser/platform are you seeing this in? (WebXR and #15393 should be Chrome on Android only)

zhudongwork commented 5 years ago

可能重复#15393

Is there some solution for this problem? This seems to be a little different from #15393 .

zhudongwork commented 5 years ago

What browser/platform are you seeing this in? (WebXR and #15393 should be Chrome on Android only)

Chrome and Firefox

jsantell commented 5 years ago

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }
zhudongwork commented 5 years ago

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work. I am using three.js to achieve volume rendering, using the methods LEBARBA provides. The first scene is rendered into a texture using WebGLRenderTarget, and the world coordinate position is stored in the RGB channel of the texture. In contrast, RenderTargetTexture is the uniforms of ShaderMaterial. The specific description and code is here.

mrdoob commented 5 years ago

@zhudongwork

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work.

Can you create a jsfiddle we can use to fix that other problem?

zhudongwork commented 5 years ago

@zhudongwork

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work.

Can you create a jsfiddle we can use to fix that other problem?

Thank you very much, my code is on the CodePen.

zhudongwork commented 5 years ago

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }

I was initially thinking about whether WebGLRenderTarget caused this problem, so I tried to modify my code to make it a one-step ray casting, and I didn't need to use WebGLRenderTarget. However, when I entered the VR mode again, I still encountered the same problem. The distance between the scenes of the left and right eyes was wrong, and the image moved along with the head. So, I was wondering if this problem was caused by THREE.ShaderMaterial using a custom vertexShader and fragmentShader in the VR mode of THREE.js.My code is on the CodePen.

zhudongwork commented 5 years ago

Is it a problem with the projectionMatrix and modelViewMatrix in the vertexShader when the value is passed in VR mode?

mrdoob commented 4 years ago

@zhudongwork any chance you can update your CodePen to use the latest version of the library?

Mugen87 commented 4 years ago

Closing. The usage of WebGLRenderTarget and WebXR is explained here: https://github.com/mrdoob/three.js/issues/18746#issuecomment-591441598.

@zhudongwork Feel free to reopen the issue if you can reproduce the problem with the latest API.