screego / server

screen sharing for developers https://screego.net/
https://app.screego.net
GNU General Public License v3.0
7.62k stars 551 forks source link

Supports audio sharing #181

Open strvert opened 3 months ago

strvert commented 3 months ago

I understand that Screego aims to remain extremely simple, and that audio support was previously closed in issue #102. However, in my daily work, I find it inconvenient not to be able to share audio when I need to demonstrate code and its results to colleagues. As a game programmer, a lot of sound is produced as a result of my code. Not being able to convey this to my colleagues is a significant drawback.

Considering the context of issue #102, I have implemented a slightly different concept, which I will explain below.

Audio Transmission Settings

image

In the browser menu, users can choose whether to share audio. If enabled, both video and audio tracks will be sent; if disabled, only the video track will be sent. In browsers that do not support audio sharing, such as Firefox, this menu will not be displayed, and only the video track will always be sent.

Audio Listening Settings

First, audio can only be listened to under the following conditions:

  1. It is not the hostStream.
  2. It is the selectedStream.
  3. The "Hear Audio" button is enabled.

The reason for 1 is straightforward—nobody wants to hear their own screen's audio twice.

For 2, the concept of this implementation is the reason. When receiving multiple screens, I couldn't think of a useful scenario where overlapping audio is helpful. Also, having many mute buttons displayed would not be simple. Therefore, by limiting the audio track used for playback to the selected stream, users can choose which audio to hear.

For 3, this is the only additional button in this implementation. With just 2, you can choose which audio to hear but cannot choose not to hear any audio at all. So, I added a button to the control menu to toggle the audio on and off. The default state is Mute. If the selected stream is your own screen or does not include an audio track, the button will automatically be disabled.

image image image

Of course, even in browsers that do not support audio sharing, such as Firefox, you can still receive audio.

I believe this approach maintains simplicity.

strvert commented 3 months ago

I expelicitly set echoCancellation and noiseSuppression to false to disable audio effects applied by the browser. I combined the MediaStreams into one and added the AudioTrack and VideoTrack to it. Additionally, the audio icon will not be displayed when there is no AudioTrack.

strvert commented 3 months ago

For consistency with other features, I added the hotkey 'a' for audio. Along with this, an issue arose from the timing of initilizing the audio element reference, so I replaced it with the more appropriate useRef.