AudioListener: Performance issue with suspended audio context and `linearRampToValueAtTime()`.

[osov]

Three.js version

• [x] r109

Browser

• [x] Chrome

OS

• [x] Windows

the problems are pretty similar:

https://github.com/mrdoob/three.js/issues/15344

https://github.com/mrdoob/three.js/issues/15422

if you create a AudioListener, add to the scene, but no sound is created, then after some time the FPS will begin to fall. The reason seems to be in the browser implementation, as the load is on the "linearRampToValueAtTime" method. If you remove the listener from the scene, then the FPS is normalized, or if you add sound and play it, the problem will also be solved. I do not know how best to solve this problem, I just report on its presence. perfomance linearRampToValueAtTime Minimum part sample:

var listener = new THREE.AudioListener(); 
camera.add( listener ); 

[Mugen87]

Can you please provide more information about this issue?

• Which browsers are affected?
• What do you mean with: "after some time"? 10 seconds, 1 min, 1 hour?
• Can you reproduce the issue with this live example? https://jsfiddle.net/nxyjaLqv/ (I can't see a framedrop even after 10 minutes).
• Can you reproduce the issue with plain Web Audio?

[osov]

Which browsers are affected?

Chrome 77.0.3865.90

What do you mean with: "after some time"? 10 seconds, 1 min, 1 hour?

Over 5 minutes.

Can you reproduce the issue with this live example? https://jsfiddle.net/nxyjaLqv/ (I can't see a framedrop even after 10 minutes).

There is no AudioListener component in this demo

Can you reproduce the issue with plain Web Audio?

I have not tried it. The problem appears if there is a AudioListener, but there is no active sound.

[Mugen87]

There is no AudioListener component in this demo

Sry, I've forgotten to click on "save". Still, I'm unable to reproduce (with Chrome 77.0.3865.90 and macOS 10.14.6 ). Here is the correct link: https://jsfiddle.net/418ko930/

I have not tried it.

Please do so and report the issue here if necessary: https://bugs.chromium.org/p/chromium/issues/list

[osov]

the problem is relevant from the very start, you do not even have to wait. perfomance I did not send a report to Google.

[Mugen87]

Sorry, I can't reproduce this on my machine. Besides, your screenshot shows the frametime, not FPS. 16.7ms (or ≈17ms) corresponds to 60 FPS.

I vote to close the issue. If anything, this needs to be reported to Chromium.

[osov]

Besides, your screenshot shows the frametime, not FPS. Are you aware of that?

Yes. under normal conditions on other demos it shows me 1-2 ms.

[Mugen87]

Yes. under normal conditions on other demos it shows me 1-2 ms.

So you have a framerate of 1000 FPS? It seems stats.js is broken on your system.

[osov]

No, 60 FPS. 1-2ms. Yes, I didn’t measure the stats correctly. 17 ms really.

[mrdoob]

So the issue here is that @osov was not understanding that 17ms ≈ 60fps, correct?

[Mugen87]

I'm not sure. The stats from his performance analysis (https://imgur.com/n7O7GZK) looked indeed strange. However, I could not reproduce this on my iMac.

[manthrax]

I am having a similar issue on windows chrome.

There's the performance graph showing linearRampToValueAtTime dominating the frame time.

If I pause the app at renderer.render() and remove the audioListener from my scene.. the frametime goes back to 60hz.

Looking at the code, I have some ideas about what could be going wrong... and that the API design might be trying to solve an impossible problem, or at least trying to automate something inherently troublesome.

[manthrax]

https://github.com/mrdoob/three.js/blob/dev/src/audio/AudioListener.js

Specifically the construct in updateMatrixWorld, used to update the listener parameters.

It seems to assume that the deltaT between previous updateMatrixWorld calls should be used as the interpolant deltaT to ramp over..

What isn't clear to me is what happens when the app/rendering is paused.. suddenly the ._clock will have a huge delta from the paused duration.. and then the update of the listener params will happen over that huge delta from the current frame?

I guess I don't understand why timeDelta should be in that equation. The context.currentTime is the current time.. so that's what time the ramp should be ramping to.. not the time of the current frame + the deltaT of the last frame. Additionally.. in the scenario where the scene is rendered multiple times for one frame, it appears that the deltaT would only be legitimate for the first render, and subsequent renders would push another ramp. In practice, since matrixWorld is cached, this probably doesn't come up, unless it gets invalidated between those renders.. but.. I think there might be a problem here, or this might be exposing a bug in chrome?

[Mugen87]

I guess I don't understand why timeDelta should be in that equation.

https://github.com/mrdoob/three.js/pull/11133#issuecomment-307882420

[FlorentMasson]

I can confirm I have this issue too (on Windows and Chrome OS) It raises so slowly it needs a while to reach 16ms on a powerful computer and then, affect framerate. Here's an update to @Mugen87's fiddle showing rendering time in ms (averaged over 5s). https://jsfiddle.net/mk1wauvp/12/ It takes about 3 minutes for me to go from 0.5ms to 1ms (the tab needs to be in foreground obviously)

[BeardScript]

Hello! I can confirm I have the same issue with PositionalAudio as well. Normal Audio has no issues it seems. With offscreen canvas it helps to make it less painful on PC but on Android is really bad in both Edge and Chrome. Firefox has persistently bad performance but not incremental and not related to this. On PC neither Firefox nor Edge show this issue, being the later the one which provides the smoothest experience overall.

I really need this so if there's something we can do on three.js end I'd be happy to help.

[Mugen87]

I have investigated this issue today and I believe I have found the root cause. There seems to be a drop in performance if linearRampToValueAtTime() is called with a suspended audio context. If the audio context is properly started with a user interaction, there is no performance issue on my system. Here are both fiddle for comparison (iMac + Chrome):

Bad: https://jsfiddle.net/mk1wauvp/12/ Good: https://jsfiddle.net/1j798645/

Notice how the "bad" fiddle produces the warning:

The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page.

@mrdoob I'm unsure how to proceed from this point. I don't know what's happening under the hood but for some reasons Chrome has problems with processing linearRampToValueAtTime() calls when the audio context is suspended. At first glance, it looks like a bug. However, we could handle this on engine level by checking this.context.state and only invoke the Web Audio API if its value is running. What do you think?

/cc @hoch

[BeardScript]

Thank you so much for looking into this @Mugen87. I can confirm that I don't see any issues in the "Good" demo (Windows + Chrome). I've done some testing as well and I've found that, at least with an Audio object, if the listener is referenced within it but is not present in the scene, there are no issues. I haven't tested this with PositionalAudio though.

[hoch]

This sounds like a bug, or something we can improve in Chrome. Could anyone file an issue on crbug.com (with Bad link above) and cc me (hongchan@chromium)?

[Mugen87]

Done: https://bugs.chromium.org/p/chromium/issues/detail?id=1106389

Sorry, I don't know how to add your email to the bug report.

[hoch]

No worries. You added Blink>WebAudio component correctly! Thank you for filing the issue and we will investigate.