benvanik / vr.js

NPAPI plugin to expose fun VR devices to Javascript.
Apache License 2.0
498 stars 105 forks source link

Mutltipass rendering has clearing issues. #19

Open AlexJWayne opened 10 years ago

AlexJWayne commented 10 years ago

I have a project that calls render 3 times per frame and composites the result. It basically renders 3 individual scenes on top of each other. In plain ol' Three.js it looks like this:

(Pardon the coffee script)

# setup renderer
@renderer = new THREE.WebGLRenderer antialias: yes
@renderer.setSize window.innerWidth, window.innerHeight
@renderer.autoClear = no # do not autoclear on each call to render

# Render a frame
render: ->
  @renderer.clear yes, yes, yes
  @renderer.render background.scene, @camera

  @renderer.clear no, yes, yes
  @renderer.render midground.scene,  @camera

  @renderer.clear no, yes, yes
  @renderer.render foreground.scene, @camera

At the start of each render, we clear all frame buffers (color, data, stencil), then we render the background. Now we clear only the depth and stencil buffers and then render the midground. And again for the foreground. We've now cleared the last frame and composited the next frame from three render calls.

Transitioning this strategy to use the OculusRiftEffect was tricky. The problem is here:

// from the end of THREE.OculusRiftEffect.prototype.render
this.renderer_.render(scene, this.eyeCamera_, undefined, true);

That last argument is the forceClear argument. And it's hardcoded to true. This means my foreground render was clearing the previous drawn scenes.

In order to fix this issue, I had to hack the library. I'm not saying this is the solution, but it worked for me.

// Added 4th `clear` argument
THREE.OculusRiftEffect.prototype.render = function(scene, camera, vrstate, clear) {
  // ... other code ...
  this.renderer_.render(scene, this.eyeCamera_, undefined, clear);
  // ...
}

And now my own render loop can now look like this:

# Render a blank scene and clear the buffer
@oculusRenderer.render emptyScene, @camera, undefined, true

# With a cleared canvas, render each layer
@oculusRenderer.render background.scene, @camera, undefined, false
@oculusRenderer.render midground.scene,  @camera, undefined, false
@oculusRenderer.render foreground.scene, @camera, undefined, false

Rendering a blank scene to control clearing is totally messy, but it seemed the only way I could figure out to get a clean clear when I wanted it. So I'm not sure what the proper API is for this case, or how to implement it, but there is clearly an issue here that should be addressed with multipass rendering.

nickoneill commented 10 years ago

I always hesitate to add another argument to a function when it'll be so infrequently changed (and I think multipass rendering is probably an advanced topic). I would rather see the addition of renderNoClear or more generic renderWithOpts {options} for cases like this.

Also: how is your framerate with three renders?

AlexJWayne commented 10 years ago

To be fair, my solution is terrible. I'm merely illustrating how I worked around problem with some obtrusive monkey patching. I just think the lib ought to make this possible somehow. Perhaps mirroring the THREE.WebGLRenderer's autoClear = false and clear(color, depth, stencil) would make more sense so that my old code could be directly copied to OculusRiftEffect.

Framerates are decent enough, as each layer is not that complex. Here's the project: http://lysertron.com/

And here's the experimental Oculus support: http://viz.lysertron.com/renderer?song=https%3A%2F%2Fs3-us-west-1.amazonaws.com%2Flysertron%2Fcc-songs%2FNovaSiberia.mp3&background=37&vr=1

Note: I don't have an Oculus, so I'm not even really sure it's working as intended...