Possible performance degradation in VR with r160 when using certain materials

[Elettrotecnica]

Description

Dear maintainers,

I am using three.js via A-Frame, on a scene closely related to the one in https://github.com/AdaRoseCannon/aframe-xr-boilerplate.

In this particular scene, the material for the big glass window in the venue is replaced on the fly by a different one (see https://github.com/AdaRoseCannon/aframe-xr-boilerplate/blob/bb5a6b97aa4bf4058dd1512b2ef3155df8a5ccc6/main.js#L276-L322).

In my scene, I have not performed this dynamical material replacement and just used the original model material. So far this has been working good enough with A-Frame versions <= 1.5.0.

When experimenting with newer A-Frame versions, I have noticed a severe sluggishness when accessing my scene in VR. At first I have blamed my custom code, but after some research and optimization attempts I could pinpoint the change in behavior to the switch to r160 in https://github.com/aframevr/aframe/commit/f06bcb6993d7b9ca04904d49d7a766c65bd3e3c7, in conjunction with the model used in the scene https://cdn.glitch.global/d29f98b4-ddd1-4589-8b66-e2446690e697/venue.glb?v=1644331843500

Reproduction steps

I have prepared 2 Glitch examples where I show the behavior before and after the switch to r160.

on three.js < r160

1. Visit https://sudden-equal-ghoul.glitch.me/ using Meta Quest 2 and Wolvic browser
2. Enter VR and look around in the direction of the big window

Expected result: decent framerate

on three.js >= r160

1. Visit https://alkaline-curse-eye.glitch.me/ using Meta Quest 2 and Wolvic browser
2. Enter VR and look around in the direction of the big window

Expected result: sluggish framerate

Notes

• replacing the material on the fly like in the original boilerplate example will mitigate the issue. This tells me the issue may be related with the type of material
• issue can be reproduced also on the native quest browser, but being more performant, this could be less noticeable
• issue can be reproduced also on Quest 3
• All versions of A-Frame using three.js >= r160 display this behavior

I attach a picture of the showing which material I am referring to in the venue model.

Code

See glitch...

Live example

• before r160
• after r160

Screenshots

Version

r159

Device

Headset

Browser

Chrome, Firefox

OS

Android

[mrxz]

The glass material is using transmission, meaning a transmission pass is rendered (once per eye). What changed in r159 is that the renderer now reports the webxr framebuffer size when in a WebXR session (https://github.com/mrdoob/three.js/pull/26905), which ends up being used as the dimensions for the transmission render target (WebGLRenderer.js#L1429-L1432). In other words, the transmission render target ends up being larger, explaining the drop in performance.

The new behaviour is more correct as the transmission render target size now relates to the actual WebXR framebuffer size. However, I noticed it's actually larger than it needs to be. The WebXR framebuffer has two views side-by-side, but the transmission pass simply takes these dimensions and renders to the full extent (meaning it ends up twice the needed width).

But even if the above would be fixed, it's likely you'd still notice a performance hit. Ultimately multiple render passes are ill-suited for (mobile/stand-alone) XR. I'd advice replacing the material in question (either at runtime or by editing the .glb file).

[Mugen87]

The WebXR framebuffer has two views side-by-side, but the transmission pass simply takes these dimensions and renders to the full extent (meaning it ends up twice the needed width).

@mrxz Would you be willing to fix this with a PR?

[Mugen87]

Also related: #22594

[mrxz]

@Mugen87 I gave it a shot, see https://github.com/mrdoob/three.js/pull/28078

The transmission pass is remarkably tricky, especially with nested render calls. The PR should fix the incorrect dimensions, prerendering the transmission pass(es) and postponing clears on the main framebuffer.

There is one point left that would improve it further and that is invalidating the depth, as only the colour information is needed. It seems that the logic for the __ignoreDepthValues flag doesn't apply when using the multisample RTT extension on Quest.

[Elettrotecnica]

@mrxz many thanks for your reply and for looking into this. My particular problem I have solved by manipulating the material on the fly like in the original scene. I am not sure I will be able to contribute further on the matter, but feel free to keep discussing here and close at your convenience :-)

[Mugen87]

This particular issue should be fixed via #28088.

Let's tackle the further performance optimizations regarding the transmission pass in https://github.com/mrdoob/three.js/issues/22594.