search

opentok / opentok-react

React components for OpenTok.js
https://www.npmjs.com/package/opentok-react
MIT License
107 stars 105 forks source link

change video to active speaker #188

Open MuhammedKhaled-95 opened 3 years ago

MuhammedKhaled-95 commented 3 years ago

I've been trying to figure out if there is any workaround to switch a div// a react component which includes the active speaker video for the currently active client.

MuhammedKhaled-95 commented 3 years ago

@aiham @ggoldens @msach22 I would totally appreciate it if you could assist me.

aiham commented 3 years ago

I haven't worked on opentok in 3 years so can't help much. From what I remember I think you might need to get access to the audio data that's coming in from the subscriber and analyse it to see if there is sufficient noise above a particular threshold. From glancing at the JS API reference I don't see any way to get a subscribers audio data. What you could do instead is use the audioSource property in OT.initPublisher https://tokbox.com/developer/sdks/js/reference/OT.html#initPublisher and pass in a custom MediaStreamTrack. Get the microphone track manually using the browser's getUserMedia function, analyse the audio data to see if there is any active speech and send that signal to all the peers on the call telling them who is currently active, either via your own custom API or using data channels. You probably need to split the track from getUserMedia and send one to your analyser and one to OT.initPublisher. Just an idea, I've never done this. Sorry I can't be any more help.

aiham commented 3 years ago

I just realised the idea above is total overkill. Listen for the audioLevelUpdated event on the Subscriber. Focus the subscriber's div which has the highest value. https://tokbox.com/developer/sdks/js/reference/Subscriber.html#events

MuhammedKhaled-95 commented 3 years ago

@aiham Thank you very much for your reply. I was able to get the corresponding Div for the active speaker from audioLevelUpdated > target.element