twilio / twilio-video-app-react

A collaboration application built with the twilio-video.js SDK and React.js
Apache License 2.0
1.8k stars 725 forks source link

When screen sharing is stopped, the audio stops as well. #786

Closed kendji closed 1 year ago

kendji commented 1 year ago

I tried to feel as if I were the user and still found it difficult to use. Reconnection is required to restore voice. I tried muting and unmuting the microphone and changing the microphone device selection, but could not recover it.

https://github.com/twilio/twilio-video-app-react/blob/6b8c19ed098350ddec43b1e68bc654f374437bfc/src/components/VideoProvider/useScreenShareToggle/useScreenShareToggle.tsx#L34

When this line is processed,

https://github.com/twilio/twilio-video.js/blob/74aa52d5b99e622d113cd69f5c82ab64fc61e1dd/lib/localparticipant.js#L613

This line unpublishes the shared screen track, but discards the audio at the same time. When I tried to share the screen again, the audio was restored, so I thought that the audio track was involved,

https://github.com/twilio/twilio-video-app-react/blob/6b8c19ed098350ddec43b1e68bc654f374437bfc/src/components/VideoProvider/useScreenShareToggle/useScreenShareToggle.tsx#L18

even though it had specified in this line that no audio was needed.

I changed my mind and tried to recover the audio with program code, but at this time there is only 'video' left in the local track. When I tried to publish only the audio, an exception occurred, indicating "Failed to execute 'getDisplayMedia' on 'MediaDevices': video must be requested".

How can this be resolved? Is it a good idea to reply to the customer with a crisp look and say, "It's just a spec"?

kendji commented 1 year ago

I checked today and the audio was still being shared after screen sharing. I do believe that the audio was lost the other day, but I will close this case as a priority to understand the latest situation.

kendji commented 1 year ago

After all, the audio feed from the web browser on the PC that started and stopped sharing has stopped.

kendji commented 1 year ago

The system disconnects once when the screen sharing is finished, and then reconnects again. It is ok. I'll close here.

kendji commented 1 year ago

I am very sorry. The cause of this matter was our predecessor.

 useEffect(() => {   if (step === Steps.deviceSelectionStep && !mediaError) {    getAudioAndVideoTracks().catch(error => {     console.log('Error acquiring local media:');     console.dir(error);     setMediaError(error);    });   }  }, [getAudioAndVideoTracks, step, mediaError]);

It seems that my predecessor added a function to this code to get the status of the data track, but because he did not wait for the processing in then() and got it as it was, there was a problem with camera flips, etc. that kept reconnecting due to a misunderstanding of the status. I realized that there are still engineers who do not understand asynchronous processing.