Closed clementroche closed 1 year ago
Do you see similar performance issues on macOS or other OS as well? There was an issue with Chromium on Windows but it has been fixed for quite a while now: https://bugs.chromium.org/p/chromium/issues/detail?id=1256340
Can you also make a test with Firefox?
On my laptop (MacBook Pro, Apple M1 Pro, Chromium-based browser) I don't see a difference, would be curious for the result in Firefox on Windows as well.
@Mugen87 @donmccurdy i'm not familiar with Firefox devTool, it's hard to tell but i would say it's not as obvious as Chrome
I see the difference using MacBook air M1 2020 but less compared to Windows
I also see a massive performance drop on my M1 macbook air between these two versions on Chrome.
I wonder if this is related to sRGB -> Linear encoding. I recently ran into an issue with updating a CanvasTexture animation and can only get reasonable performance if I set the texture encoding to Linear. Here's a minimal example: https://codesandbox.io/s/2d-canvas-texture-encoding-performance-iphunc?file=/src/App.js
@toddka I see no FPS difference no matter using sRGB or linear-srgb. I'm on Chrome 114.0.5735.90, macOS 13.4 and a M2 Pro. Can you please quantify the difference in performance so we can compare your FPS values?
I'm on Chrome 114.0.5735.90, M1 2020 Macbook Air, MacOS Ventura 13.0.1.
Canvas width and height seem significant. If I decrease the canvas size FPS increases. I'm seeing the FPS drop from stable 60 FPS to 23-28 FPS when switching to texture.encoding = THREE.sRGBEncoding
with a 1725x1280 canvas.
Let me know if I can provide anything more concrete in terms of performance numbers or repro. I'm also not sure if the issue in my sandbox is related to this, so happy to create another issue if requested! I've run into OP's video performance issue as well.
I've made a more intense test case and now I can confirm performance issues with hardware sRGB decode in Chrome. The following fiddles update a 4096x4096 canvas texture per frame:
(A) CanvasTexture
with THREE.LinearSRGBColorSpace
: https://jsfiddle.net/8p7tk5y9/3/
(B) CanvasTexture
with THREE.SRGBColorSpace
: https://jsfiddle.net/8p7tk5y9/2/
(A) runs with 60 FPS, (B) with 7 FPS. The entire tab becomes quite slow with (B).
I don't see this kind of performance difference with Firefox. Both fiddles run with 23 FPS.
Safari slows down from 15 to 8 FPS.
@donmccurdy Can you reproduce on your MacBook?
Chrome/114.0.5735.90 Linux ANGLE (AMD, AMD Radeon RX 460 Graphics (polaris11 LLVM 16.0.4), OpenGL 4.6 (Core Profile) Mesa 23.0.3)
(A) 60 FPS (B) 4 FPS
DEV Chrome/115.0.5790.3 Linux ANGLE (AMD, Vulkan 1.3.238 (AMD Radeon RX 460 Graphics (RADV POLARIS11) (0x000067EF)), radv-23.0.3)], DRIVER_VENDOR=AMD, DRIVER_VERSION=23.0.3
(A) 60 FPS (B) 60 FPS
Interesting. I've made a test with Chrome Canary (116.0.5809.0) and both fiddles run with 60 FPS!
So it seems that the issue was fixed in Chrome 115?
I'm still seeing this issue across wide variety of hardware, browser and OS versions, including Chrome 116:
Tested both of @Mugen87's fiddles:
Chrome 116 and 115 – Mac OS Ventura, M1 Max, 16 inch
Chrome 116 and 115 – Mac OS Catalina, Intel i9, AMD Radeon Pro 5500M 8GB
Edge 112, Chrome 116 and 115 – Windows 10, Intel i5, Nvidia GTX 1650
This issue is currently blocking us from shipping a new feature, let us know how we can help.
For what it's worth:
Tested both of @Mugen87's fiddles again:
Firefox performs bad, but consistently bad, no difference between those fiddles:
Firefox 109, Mac OS Ventura, M1 Max, 16 inch:
Firefox 109, Mac OS Catalina, Intel i9, AMD Radeon Pro 5500M 8GB
Firefox 109, Windows 10, Intel i5, Nvidia GTX 1650:
Safari 16.3, Ventura M1 Max 16 inch:
Safari 13.1.3, Catalina, Intel i9, AMD Radeon Pro 5500M 8GB:
I'm also seeing that something broke with sRGBColorSpace between three 151 and 152:
Forked the sRGB canvas fiddle:
~~three 152.0: https://jsfiddle.net/dfeehrer/grmv47js/3/ ~~
- 8-10 FPS on M1 Max Mac, Chrome 115 and 116
three 151.3: https://jsfiddle.net/dfeehrer/4zegy02k/13/
- 60 FPS FPS on M1 Max Mac, Chrome 115 and 116
Nothing broke. r151
does not know about colorSpace
so when you assign it, it is not processed. Meaning the texture stays in linear-srb.
Here is the fiddle using the deprecated encoding
property. It produces the same performance issues: https://jsfiddle.net/x7nkbcd1/
It is strange that Chrome >= 115 fixes the issue for some setups, but not for all. Probably worth reporting the issue to the Chromium bug tracker.
Nothing broke.
r151
does not know aboutcolorSpace
so when you assign it, it is not processed. Meaning the texture stays in linear-srb.
@Mugen87 ah you're right, forgot about the name change
Is it possible to check if the ANGLE WebGL backend is enabled or disabled?
@dfeehrer Do you mind typing chrome://gpu/
in the address bar of Chrome on one of your systems and copy the report in this issue?
Here's the report for the M1 Max Mac, Chrome 115:
on Chrome/116.0.5803.2 on Linux Vulcan is enabled as default on this system when i disable vulkan with chrome://flags same problem with 4 fps (B) is reproducible.
[ANGLE (AMD, AMD Radeon RX 460 Graphics (polaris11 LLVM 16.0.4), OpenGL 4.6 (Core Profile) Mesa 23.1.1)], DRIVER_VENDOR=Mesa, DRIVER_VERSION=23.1.1 ACTIVE
Another data point. Windows 11. Chrome Version 116.0.5809.2 (Official Build) canary (64-bit). 3070 with Ryzen 5 5600x. Measured GPU and CPU load with HWiNFO. Only 4-7 fps on the sRGB texture example. 165 fps on linear. Increased CPU and GPU load. Is it possible the color conversion is happening on the CPU? That seems unlikely but CPU load increases by a larger percentage compared to GPU load.
On the M1 Mac this issue only shows up in the default OpenGL ANGLE backend, not in the Metal ANGLE backend
@dfeehrer @toddka Would one of you be willing to report this issue at the Chromium bug tracker instead? You can use my fiddles from above for reproduction test cases. It would be good if someone reports the issue who has still issues with Chrome 115/116.
The URL is: https://bugs.chromium.org/p/chromium/issues/list
BTW: When filing the issue, please add the chrome://gpu/
report as an attachment. I could not see something suspicious when investigating @dfeehrer's report but a Chromium dev can analyze the report better than myself. The component of the bug (it's category) would be Blink>WebGL
.
Chromium bug report: https://bugs.chromium.org/p/chromium/issues/detail?id=1451195
Thanks! I've added a comment to highlight that the issue is related to the WebGL format SRGB8_ALPHA8
. That is the internal format a sRGB canvas texture is using. I suppose browsers do not handle this format properly in all cases.
@Mugen87 awesome thanks! Really appreciate all of the helping addressing this.
This seems like it would be a fairly common use case. I wonder how many people are running projects on Metal or Vulcan and are unaware that their users are hitting this performance issue. Is there a workaround we could suggest for the time being given that browser bugs may take a long time to fix?
One idea I have is to write a custom shader for the canvas texture that does the linear -> sRGB encoding. I'm not sure if this is possible or nontrivial (I'm a complete WebGL/GLSL noob), but I'd be willing to work on it and post a solution here for others while we wait for a Chromium fix.
I think the overhead only becomes a problem if you updating texture data per frame. TBH, I would like to wait for the feedback of the Chromium team first and see how fast it can be fixed.
I would prefer to not add inline decoding back. Devs how still want it can keep their textures in linear-srgb and implement the inline decode via Material.onBeforeCompile()
.
I'm experiencing this issue with video textures and WebGPU renderer on Chrome Windows. cpu usage 20%. On Firefox cpu usage is 1% and gpu 2-3% it was surprising the difference. . On Canary 3-5% cpu but 20-40% gpu.
I have to set the colorspace to LinearSRGBColorSpace to fix a gamma problem when using webgpu.
https://danrossi.github.io/three-webgpu-renderer/tests/webgpu_video_panorama_equirectangular.html https://danrossi.github.io/three-webgpu-renderer/tests/webgl_video_panorama_equirectangular.html
@danrossi Nit: Unfortunately, your post at the Chromium bug tracker is unrelated since WebGPU issues are tackled in completely different Chromium components (Blink>WebGPU
vs Blink>WebGL
).
My issue #26266 is related to this for WebGLRenderer but it might be causing the same problem for WebGPURenderer. WebGPURenderer is 20% cpu usage, the use of SRGB8_ALPHA8
produces 10% cpu in webgl and when using RGBA8
it's 1-2%. With video textures.
Video textures for WebGL has to be set to THREE.SRGBColorSpace. Unless this function is disabled in the shader. `linearToOutputTexel which I've done so in a shadermaterial
I've recently come across this issue when trying to render a 4K video in Three.js VR using an Oculus Quest 1. Both my Windows Chrome browser and the Oculus browser (which is Chromium based) were running at a max of 15fps. The quick solution was to set the Three.js WebGLRenderer outputColorSpace to LinearSRGBColorSpace
and leaving the VideoTexture
colorSpace as default (three.js r153). I'm only rendering a VideoTexture and nothing else so this solution works fine for me in the meantime.
That is the one. Mine is cpu related than dropping frames. I have even higher cpu issues with WebGPU and drops frames every 30 seconds of playback and fan ramps up.
With WebGL using LinearSRGBColorSpace produces a wierd gamma issue because of a conversion in the shader. It requires srgb colorspace setting like webgpu does now. One of my usages is a video grid rendering multiple video textures so performance is needed.
Here is my work around for now. Using a shadermaterial with the shader function causing the gamma issue removed when using linearsrgb
I was constantly advised my problem was not related so made a new ticket. Its when SRGB8_ALPHA8 is used as the internal format for the video texture.
https://bugs.chromium.org/p/chromium/issues/detail?id=1456882&q=SRGB8_ALPHA8&can=2 https://github.com/mrdoob/three.js/blob/aef215178070679b8407832a9a71123978807456/src/renderers/webgl/WebGLTextures.js#L172
It would be great when affected users could stare the Chromium bug so it becomes more visible (and hopefully gets a higher priority).
https://bugs.chromium.org/p/chromium/issues/detail?id=1451195
Devs how still want it can keep their textures in linear-srgb and implement the inline decode via Material.onBeforeCompile().
Here is a live example that implements the workaround: https://jsfiddle.net/em6vasdg/3/
I've updated my test and seems to work. No color issue using linear colorspace. https://github.com/danrossi/three-webgpu-renderer/blob/master/tests/webgl_video_panorama_equirectangular_nosrgb.html#L121
They are looking at my own ticket which is not frame dropping related but cpu. https://bugs.chromium.org/p/chromium/issues/detail?id=1456882&q=SRGB8_ALPHA8&can=2
I've recently come across this issue when trying to render a 4K video in Three.js VR using an Oculus Quest 1.
/ping @rcabanier
WebGPU issue is much worse in the other ticket. The chrome people reported 35-40% cpu but my Ryzen produces 20% cpu. And not sure if it's a chrome problem yet With WebGL Oh no this isn't good. I just rechecked firefox and producing the same cpu usage with srgb as Chrome. And low cpu usage with the linear work around. So maybe I wasted my time reporting it to Chrome. I'll try and make a vanilla webgl texture test to see what is going on with the format.
high cpu in Firefox - https://danrossi.github.io/three-webgpu-renderer/tests/webgl_video_panorama_equirectangular.html low cpu in Firefox - https://danrossi.github.io/three-webgpu-renderer/tests/webgl_video_panorama_equirectangular_nosrgb.html
My chrome issue got merged into the one referenced here as its the same problem. I found a work around but for video. I'm about to make vanilla webgl texture tests with the different formats set. If you use video callback instead of 60fps animation it uses less cpu.
I have a throttled animation frame solution as fallback in my animation
https://github.com/danrossi/three-webgpu-renderer/blob/master/src/util/VideoAnimation.js#L20
UPDATE:
I made a bunch of vanilla webgl tests. SRGB format at 30fps is 1-5%. at 60fps its 14% and chrome freezes. I have to update chrome ticket.
https://danrossi.github.io/three-webgpu-renderer/tests/webgl_vanilla_srgb.html https://danrossi.github.io/three-webgpu-renderer/tests/webgl_vanilla_srgb_60fps.html
with the other non srgb format its low cpu at 60fps
https://danrossi.github.io/three-webgpu-renderer/tests/webgl_vanilla_nosrgb_60fps.html
Is it worth considering adding this shader decoding (back) into the ThreeJS codebase?
Without it, most new and existing users aren't going to know that they need to add an obscure shader found in this issue, in order to make sure videos have acceptable performance in ThreeJS.
Claims to have been fixed in Chrome Canary on Jul 9th. Need to check it now.
Update:.Nothing changed.
I just checked Chrome canary. And I still get 10-13% cpu and 6-7% gpu usage with video textures.
No SRGB fix 0.7-1% cpu and 2.1% gpu usage
I've created a custom material module for this purpose with tests. I reported to chrome the resource issue with a vanilla example but got glazed over. It's when this is used for texture render of video
gl.texImage2D( gl.TEXTURE_2D, 0, gl.SRGB8_ALPHA8, gl.RGBA, gl.UNSIGNED_BYTE, video);
https://github.com/danrossi/three-webgpu-renderer/blob/master/build/linearsrgb-material.module.js
example bundling
https://github.com/danrossi/three-troika/blob/main/three.rollup.config.js#L113
I noticed something was reverted which will be hard to keep builds future proofed. I'm sorry I just checked Canary with vanilla texture render and cpu is low. It's in three somewhere like the webgpu issue.
vanilla seems ok to me although no srgb vanilla is less cpu which I have updated the chrome ticket about. https://danrossi.github.io/three-webgpu-renderer/tests/webgl_vanilla_srgb.html
https://danrossi.github.io/three-webgpu-renderer/tests/webgl_vanilla_nosrgb.html
cpu jumps up in three. I'm not sure what the issue is but they may have fixed it.
https://danrossi.github.io/three-webgpu-renderer/tests/webgl_video_panorama_equirectangular.html
The srgb cpu usage has dropped iin Canary but there is an obvious difference using RGBA8
Description
Since 0.152.0 VideoTexture takes a lot of GPU ressources, the larger the video the slower it gets.
Windows 11 NVIDIA GeForce RTX 3060 Ti
Maybe related to this commit https://github.com/mrdoob/three.js/pull/25752/commits cc @donmccurdy
Reproduction steps
Open Chrome Devtools performance tab in v0.152.2 and you gonna notice GPU is busy
Code
See live examples
Live example
v0.152.2 https://codesandbox.io/s/three-0-152-2-videotexture-mj2y8m (bad performance) v0.151.3 https://codesandbox.io/s/three-0-151-3-videotexture-1et5sq (good performance)
Screenshots
Version
">= 0.152.0"
Device
No response
Browser
Chrome
OS
Windows