search

twilio / twilio-video-app-react

A collaboration application built with the twilio-video.js SDK and React.js
Apache License 2.0
1.81k stars 728 forks source link

Change shared tab on the fly? #700

Closed haffla closed 2 years ago

haffla commented 2 years ago

Question Chrome usually shows a button "Share this tab instead" when sharing a tab. Any idea why this button doesn't show?

Screenshot 2022-06-10 at 12 42 58

timmydoza commented 2 years ago

Hi @haffla - thanks for the question!

This is a great question. I can see in Google Meet that there is an extra "Share this tab instead" button when sharing a tab.

I dug into this quite a bit, and unfortunately I'm not quite sure that it's easy to add this to our Twilio Video App. In this app, we use the navigator.media.getDisplayMedia() api to capture the user's screen. This function can accept a 'constraints' object which can change various properties of the screen capture, like resolution, or frame rate. In my testing, I could not find any constraint that enabled the "Share this tab instead" button.

So how does it work in Google Meet?

Well I don't think that Google Meet uses the same same navigator.media.getDisplayMedia() API. It looks like Google Meet uses the chrome.desktopCapture.chooseDesktopMedia() API to capture a user's screen. It's important to note that this chrome.desktopCapture.chooseDesktopMedia() API is not available to this application - it is only available to Chrome Extensions (more info here: https://developer.chrome.com/docs/extensions/reference/desktopCapture).

So how does Google Meet use an API that is only available to Chrome Extensions, when there is no Google Meet extension installed.

It looks like Chrome has an "invisible" extension installed for "Google Hangouts", which Google Meet uses. You can see it if you go to chrome://inspect/#extensions while in a Google Meet meeting:

image

If you click "inspect" you can even look at the source code for this extension, and you'll see that they use chrome.desktopCapture.chooseDesktopMedia() instead of navigator.media.getDisplayMedia() (this extension is also how Google Meet is able to display your CPU usage in the Troubleshooting menu. The chrome.system.cpu.getInfo() API is used in the Google Hangouts extension).

So it's my suspicion that the "Share this tab instead" button is available when using Chrome's chooseDesktopMedia() API, but not when you are using the navigator.media.getDisplayMedia().

It would be possible to implement this feature in this app - but it would be significantly more work than our current implementation with navigator.media.getDisplayMedia(), but it's possible! It would also require the user install a Chrome Extension. But because of the extra effort and complexity, we won't be adding this feature to the React Video App.

If you're interested in trying to build a Chome Extension for screen sharing, you can see an example in our documentation here: https://www.twilio.com/docs/video/screen-capture-chrome#chrome-71-build-a-screen-share-extension. We also have similar blog posts about the topic here: https://www.twilio.com/blog/2018/01/screen-sharing-twilio-video.html and here: https://www.twilio.com/blog/screen-sharing-javascript-twilio-programmable-video.

Thanks again for the question - and please let us know if there's anything else we can help with!

haffla commented 2 years ago

Amazing investigation @timmydoza. Thanks a bunch. This is really super helpful. I'm gonna dig into it.

haffla commented 2 years ago

Just wanted to let you know that I've managed to do this and it's indeed true. If you use chrome.desktopCapture.chooseDesktopMedia you'll get that button.