VideoTexture: Slow performance on Quest Browser in non-XR mode.

[cabanier]

Description

The WebGPURenderer is a lot slower on Quest. People are reporting that their experiences are running poorly.

In the console I see:

WebGPU canvas configured with a different format than is preferred by this device ("rgba8unorm"). This requires an extra copy, which may impact performance.
webgpu_video_panorama.html:1 entries[2] is not an ExternalTexture when the layout contains an ExternalTexture entry.
 - While validating [BindGroupDescriptor "bindGroup_object"] against [BindGroupLayout (unlabeled)]
 - While calling [Device].CreateBindGroup([BindGroupDescriptor "bindGroup_object"]).

webgpu_video_panorama.html:1 entries[2] is not an ExternalTexture when the layout contains an ExternalTexture entry.
 - While validating [BindGroupDescriptor "bindGroup_object"] against [BindGroupLayout (unlabeled)]
 - While calling [Device].CreateBindGroup([BindGroupDescriptor "bindGroup_object"]).

webgpu_video_panorama.html:1 [Invalid BindGroup "bindGroup_object"] is invalid.
 - While encoding [RenderPassEncoder (unlabeled)].SetBindGroup(0, [BindGroup], 0, ...).

webgpu_video_panorama.html:1 [Invalid CommandBuffer from CommandEncoder "renderContext_0"] is invalid.
 - While calling [Queue].Submit([[Invalid CommandBuffer from CommandEncoder "renderContext_0"]])

Reproduction steps

1. open browser on Quest 2 or higher
2. browser to https://threejs.org/examples/?q=pan#webgpu_video_panorama
3. observe that the movie plays at 3-5fps

Version

dev

Device

Quest 2 and higher

Browser

Quest Browser

OS

Quest v67

[aardgoose]

You can test if setting the preferred canvas format helps with speed if you edit the line:

https://github.com/mrdoob/three.js/blob/ab8f885327fc1423418f3fb96fe4d01057fb4e5b/src/renderers/webgpu/WebGPUBackend.js#L113

And replace 'GPUTextureFormat.BGRA8unorm' with 'GPUTextureFormat.RGBA8unorm', which should eliminate the first warning message.

This default format type should really be set using the devices preferred format obtained from navigator.GPU.getpreferredcanvasformat() at some point,

The other errors are also seen on PC type devices when starting video display (at least on my laptop).

[Mugen87]

For comparison:

Prod: https://threejs.org/examples/?q=pan#webgpu_video_panorama Dev with https://github.com/mrdoob/three.js/pull/28873: https://rawcdn.githack.com/mrdoob/three.js/b4090aab0c21450e17f8537fdbca099b4d6d7f13/examples/index.html#webgpu_video_panorama Dev without https://github.com/mrdoob/three.js/pull/28873: https://rawcdn.githack.com/mrdoob/three.js/45498d0255abe5da8d3ff95c117ccab594c49b8f/examples/index.html#webgpu_video_panorama

[Mugen87]

The video texture related warnings have been already reported here: #27690

[Mugen87]

@aardgoose It seems the dev build with https://github.com/mrdoob/three.js/pull/28873 now produces a black screen with all WebGPU examples on Quest Browser.

[Mugen87]

After some more testing the performance issue on Quest does not affect all examples. https://threejs.org/examples/webgpu_skinning runs smoothly on my Quest 2.

The issue seems to be related on video textures, only. I suspect solving https://github.com/mrdoob/three.js/issues/27690 will also solve the Quest problem.

[aardgoose]

@aardgoose It seems the dev build with #28873 now produces a black screen with all WebGPU examples on Quest Browser.

Maybe the browser is not reporting a correct canvas format? Looks like that #28873 needs reverting, it would be interesting to see what getPreferredCanvasFormat() is returning in that case.

As the video errors appear in Chrome on my laptop with no performance problems, the performance issues may be elsewhere with video.

[Mugen87]

New test link with https://github.com/mrdoob/three.js/pull/28894.

https://rawcdn.githack.com/mrdoob/three.js/abce0ba83afa7ef5d4bf5001d192c66e44b754a8/examples/index.html#webgpu_video_panorama

[Mugen87]

Unfortunately, even with https://github.com/mrdoob/three.js/pull/28894 the example is still slow on a Quest 2 :(.

[aardgoose]

I didn't expect that to improve performance. Does the equivalent WebGL example 'webgl_video_panorama_equirectangular' perform badly too? Do are able to get a devtools performance profile?

[Mugen87]

For testing:

https://threejs.org/examples/#webgl_video_panorama_equirectangular https://webgpufundamentals.org/webgpu/webgpu-simple-textured-quad-external-video.html

[Mugen87]

Okay, after some more testing I can confirm that webgl_video_panorama_equirectangular also runs slow in the Quest Browser. In fact any demos using THREE.VideoTexture run slow.

I've also tested a non-three.js WebGPU video example from WebGPU fundamentals and it runs slow in the Quest Browser as well.

So this issue seems unrelated to WebGPURenderer.

[cabanier]

Okay, after some more testing I can confirm that webgl_video_panorama_equirectangular also runs slow in the Quest Browser. In fact any demos using THREE.VideoTexture runs slow.

Interesting! I'll ask someone on my team to investigate

[Mugen87]

One additional observation:

When opening webxr_vr_video in Quest Browser, the video texture runs slow. But when entering the XR session, it runs just fine. So it seems an issue with viewing the content in non-XR mode.

[cabanier]

I've seen performance issues lately on the 2D side. Turning on passthrough surprisingly helps.