search

nextcloud / spreed

🗨️ Nextcloud Talk – chat, video & audio calls for Nextcloud
https://nextcloud.com/talk
GNU Affero General Public License v3.0
1.61k stars 429 forks source link

Automatic selection of audio device? #7461

Open nickvergessen opened 2 years ago

nickvergessen commented 2 years ago

In the planing meeting the question/idea came up if it would be possible to "magically" pick the right audio device.

Suggested idea was something like:

We recognize input when the speaking happens during a call, so should be possible to do this too?

If no input is recognized we can show some hints for the user:

nickvergessen commented 2 years ago

Not sure if that is totally possible, but raised the issue to collect feedback. cc @danxuliu @marcoambrosini what do you think, any ideas?

danxuliu commented 2 years ago

In the past Firefox did not support getting a stream from a device if there was already an active stream from another device. I have checked with latest version and that is now possible, so it should be possible to analyze the input from all microphones at the same time to find out which one is working (and disable the feature in older Firefox versions).

However... what if more than one device has input? For example, in my case I have a device for the microphone, and another device for the noise reduction node from PulseAudio. The noise reduction device should be the one used, but both record audio and thus it would not be possible to automatically know which one is the right one. In other cases I guess that you can have two physical microphones recording your voice at the same time. In the end the user will have to manually select a device from the list, so it may not change much from how it is currently done. I guess that it would depend on how frequently it happens that there are several microphones and only one of them works.

Besides that, another problem with automatic selection could be to miss the user expectations (even if it is not the fault of the automatic selection). For example, a user has an external microphone and an internal laptop microphone, but the user is not aware that the external microphone is muted at a system level. The user tells Talk to select a microphone, but Talk only gets input from the internal laptop microphone, so that is the one selected. The user could end frustrated because Talk did not select the "right" microphone. And after finding that the "right" microphone was not working... the user could also blame Talk for not warning her during the automatic selection :-P Anyway maybe this could be addressed with the UI by setting the right expectations.

In any case, from a technical point of view it should be possible to get audio streams from all the available devices, check their volume for some seconds, and known which ones were muted and which ones were not.