WebGLRenderTarget has some problems in VR mode

[zhudongwork]

Description of the problem

I am trying to render a scene into another scene using WebGLRenderTarget in VR mode. I created two scenes - mainScene and bufferScene. I draw a red cube in bufferScene, then render the bufferScene to the texture bufferTexture using WebGLRenderTarget . bufferTexture as the texture of mainScene. When I enter VR mode and turn the head to observe the entire scene, mainScene is fine, but the cube in bufferScene is rendered by split screen, and it will follow my head movement. Generally speaking, in the vr scene, the object should not follow the head movement. What caused this situation? I have put the code in codeopen.

var mainScene = new THREE.Scene();
mainScene.position.set(0,0,0);
var width = window.innerWidth;
var height = window.innerHeight;
var camera = new THREE.PerspectiveCamera( 70, window.innerWidth / window.innerHeight, 1, 1000 );
camera.position.z = 10;

var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
renderer.vr.enabled = true;
document.body.appendChild( renderer.domElement );
document.body.appendChild( WEBVR.createButton( renderer ) );

//This is where we create our off-screen render target
//Create a different scene to hold our buffer objects
var bufferScene = new THREE.Scene();
//Create the texture that will store our result
var bufferTexture = new THREE.WebGLRenderTarget( window.innerWidth, window.innerHeight,
                                { minFilter: THREE.LinearFilter,
                                    magFilter: THREE.NearestFilter});

//Let's create a red box
var redMaterial = new THREE.MeshBasicMaterial({color:0xF06565});
var boxGeometry = new THREE.BoxGeometry( 5, 5, 5 );
var boxObject = new THREE.Mesh( boxGeometry, redMaterial );
boxObject.position.z = -10;
bufferScene.add(boxObject);//We add it to the bufferScene instead of the normal scene!

////////////////////////////Now we use our bufferTexture as a material to render it onto our main scene
var boxMaterial = new THREE.MeshBasicMaterial({map:bufferTexture.texture});
var boxGeometry2 = new THREE.BoxGeometry( 5, 5, 5 );
var mainBoxObject = new THREE.Mesh(boxGeometry2,boxMaterial);
mainBoxObject.position.set(0,0,-10);
mainScene.add(mainBoxObject);

///And a blue plane behind it
var blueMaterial = new THREE.MeshBasicMaterial({color:0x7074FF})
var plane = new THREE.PlaneBufferGeometry( window.innerWidth, window.innerHeight );
var planeObject = new THREE.Mesh(plane,blueMaterial);
planeObject.position.z = -15;
mainScene.add(planeObject);

//Render everything!
function render() {
  renderer.setAnimationLoop( render );
  renderer.render(bufferScene,camera,bufferTexture,true);
  renderer.render( mainScene, camera );

}
render();

Three.js version

• [ ] Dev
• [x] r100
• [ ] ...

Browser

• [x] All of them
• [ ] Chrome
• [ ] Firefox
• [ ] Internet Explorer

OS

• [ ] All of them
• [x] Windows
• [ ] macOS
• [ ] Linux
• [ ] Android
• [ ] iOS

Hardware Requirements (graphics card, VR Device, ...)

[Mugen87]

Possible duplicate of #15393

[jsantell]

What browser/platform are you seeing this in? (WebXR and #15393 should be Chrome on Android only)

[zhudongwork]

可能重复＃15393

Is there some solution for this problem? This seems to be a little different from #15393 .

[zhudongwork]

What browser/platform are you seeing this in? (WebXR and #15393 should be Chrome on Android only)

Chrome and Firefox

[jsantell]

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }

[zhudongwork]

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work. I am using three.js to achieve volume rendering, using the methods LEBARBA provides. The first scene is rendered into a texture using WebGLRenderTarget, and the world coordinate position is stored in the RGB channel of the texture. In contrast, RenderTargetTexture is the uniforms of ShaderMaterial. The specific description and code is here.

[mrdoob]

@zhudongwork

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work.

Can you create a jsfiddle we can use to fix that other problem?

[zhudongwork]

@zhudongwork

I tried it according to your suggestion and my problem was solved. But when I used this solution for my other problem, it didn't work.

Can you create a jsfiddle we can use to fix that other problem?

Thank you very much, my code is on the CodePen.

[zhudongwork]

I misread before. The render target is getting stereoscopic rendering because VR is enabled, which synthesizes the stereo camera. Try disabling and re-enabling VR when rendering the buffer scene:

     function render() {
          renderer.setAnimationLoop( render );
          renderer.vr.enabled = false;
          renderer.render(bufferScene,camera,bufferTexture,true);
          renderer.vr.enabled = true;
          renderer.render( mainScene, camera );
        }

I was initially thinking about whether WebGLRenderTarget caused this problem, so I tried to modify my code to make it a one-step ray casting, and I didn't need to use WebGLRenderTarget. However, when I entered the VR mode again, I still encountered the same problem. The distance between the scenes of the left and right eyes was wrong, and the image moved along with the head. So, I was wondering if this problem was caused by THREE.ShaderMaterial using a custom vertexShader and fragmentShader in the VR mode of THREE.js.My code is on the CodePen.

[zhudongwork]

Is it a problem with the projectionMatrix and modelViewMatrix in the vertexShader when the value is passed in VR mode?

[mrdoob]

@zhudongwork any chance you can update your CodePen to use the latest version of the library?

[Mugen87]

Closing. The usage of WebGLRenderTarget and WebXR is explained here: https://github.com/mrdoob/three.js/issues/18746#issuecomment-591441598.

@zhudongwork Feel free to reopen the issue if you can reproduce the problem with the latest API.