voxeet / voxeet-sdk-web

The Dolby.io Communications SDK for Web.
https://www.npmjs.com/package/@voxeet/voxeet-web-sdk
Other
4 stars 2 forks source link

VoxeetSDK.conference.isSpeaking not working consistently #24

Closed TheCodeTherapy closed 11 months ago

TheCodeTherapy commented 1 year ago

According to the Web SDK docs, the isSpeaking method should have its callback value set to true if the audioLevel of a participant is above 0.2.

Please consider the following example:

for (const [, participant] of VoxeetSDK.conference.participants) {
  VoxeetSDK.conference.isSpeaking(participant, (value: boolean) => {
    console.log(participant.id, value);
  });
}

The code above is repeatedly (3 times per second) called during the conference as per the instructions on the documentation.

With the above implementation, value is console logged as true just for a very short period of time (less than a second) even though I keep speaking with the same volume on the microphone. Only after a few seconds of silence in the microphone, if I speak again, does the same behavior happen.

I'm assuming the expected behavior would be for value to be true as long as I keep speaking loudly on the microphone, and that's not what happens (it's reset to false even though I haven't stopped speaking or lowered my voice level).

I also wrote some tests replacing isSpeaking by the audioLevel method in the example above and typing the value in the callback function as a number, to check if the value returned would be greater than 0.2 as the documentation references. The same behaviour occur: I see a value greater than 0 just for a very brief period of time (less than a second) and it becomes zero right away even though I keep emitting the same amount of volume to the microphone.

Is there something I should be doing differently to reliably identify if a participant is speaking on the microphone? Are there alternative methods so I can have a reliable and real-time source of truth in regards to participants that have sound detected on their microphones?

Thanks.

FabienLavocat commented 1 year ago

Here is an example on how we are using the isSpeaking within a conferencing application: https://github.com/voxeet/voxeet-uxkit-react/blob/65e8a4f73fcc536c2a50dafd6ef62411c73fb5c9/src/app/actions/ActiveSpeakerActions.js#L13 We have implemented some kind of buffer to make the transition from speaking / not speaking smoother.