Open PVince81 opened 4 years ago
Things I've tried so far that did not work:
I ran the performance tool from Chromium but nothing special was reported there.
Hmm I also see a process "Web Content" which might be Firefox, and that one has 80% CPU also.
I've tried the following hack to close the audio and video tracks when muting (similar to https://github.com/nextcloud/spreed/issues/4008#issuecomment-706185774):
diff --git a/src/utils/webrtc/simplewebrtc/localmedia.js b/src/utils/webrtc/simplewebrtc/localmedia.js
index 35c1563a9..dba1ba160 100644
--- a/src/utils/webrtc/simplewebrtc/localmedia.js
+++ b/src/utils/webrtc/simplewebrtc/localmedia.js
@@ -600,7 +600,11 @@ LocalMedia.prototype._setAudioEnabled = function(bool) {
this.localStreams.forEach(function(stream) {
stream.getAudioTracks().forEach(function(track) {
- track.enabled = !!bool
+ if (!bool) {
+ track.stop()
+ } else {
+ track.enabled = !!bool
+ }
})
})
}
@@ -609,7 +613,11 @@ LocalMedia.prototype._setVideoEnabled = function(bool) {
this.localStreams.forEach(function(stream) {
stream.getVideoTracks().forEach(function(track) {
- track.enabled = !!bool
+ if (!bool) {
+ track.stop()
+ } else {
+ track.enabled = !!bool
+ }
})
})
}
And then clicked the mute buttons and video hide buttons 2-3 times to trigger the routine, and end up with the muted/hidden state.
The CPU usage has reduced below 10% now (and my laptop fans are resting).
So it seems it's the WebRTC / audio / video stream bits that are causing high cpu, even when disabled. Not sure if those can be optimized.
I'm starting to see a benefit to actually stopping those streams, not only for CPU usage but also for https://github.com/nextcloud/spreed/issues/4008
Another try where I close the video stream but leave audio open but disabled: both browsers report around 30 % CPU usage.
I suspect that the browsers might still be recording audio and video but discarding the data. Pulseaudio reports that both browsers have an active RecordStream. However if I close the stream nothing appears there.
If I block camera and microphone permissions, the CPU usage drops below 5%. This confirms that the CPU usage is likely related to the audio and video streams.
I see there's also a cloned audio track to detect speaking when mute, to show the hint bubble. Not sure if that cloned track is causing additional overhead.
I checked Jitsi yesterday and noticed that when disabling video, the video stream is stopped completely. But for audio the red microphone icon in Firefox was still there, I guess it's for detecting if someone is speaking.
Disabling video would already significantly reduce CPU usage, so I think disabling video and keeping audio would be a good tradeoff.
I had another look at the code and tried to (ab)use _handleVideoInputIdChanged()
in localmedia.js as I saw that this method is re-creating the video stream. The result only worked once. I'll need to invest more time to understand better how the stream juggling (localStreams
array), etc is working in more detail.
To confirm that it's not only a Linux thing, with @Ivansss we tested on a Mac and can confirm that CPU usage stays the same when disabling video in the current code. However when using the hack from https://github.com/nextcloud/spreed/issues/4355#issuecomment-707034460 (the video path), CPU usage drops by 10%.
cc @danxuliu as you seem the most familiar with this code
Steps to reproduce
Expected behaviour
No or low CPU usage
Actual behaviour
Chromium uses around 30% on my machine.
Talk app
Talk app version: 11.0.0 (git master 199c7d84f720b578aa76ba9480f03058b40e1ac4)
No custom servers
Browser
Microphone available: yes
Camera available: yes
Operating system: openSUSE Tumbleweed 20201007
Browsers: Firefox 80.0, Chromium 85.0.4183.121
Browser log
Server configuration
Nextcloud 21.0.0 alpha git master b76698c547bcaf22857f8ec67a8567399520f73a
Runs on the same machine.