[Bug] Window screensharing low FPS, no audio in Window screensharing

[Daki0082]

Describe the bug Every time I share my screen on Dorion, my friends say that my game runs at 5 FPS. And when my game is in full screen, my friends only see a still image of my game. They only see my game moving when I tab out of the fullscreen game, again at 5FPS. This only happens on Dorion, not on the normal Discord app.

To Reproduce

• Open Dorion
• Call a friend or join a voice channel
• share your screen

System Info (please complete the following information):

• OS: Windows 11
• Version: happens on all Dorion versions

[SpikeHD]

Looks like this might be due to this bug with the VP9 codec. I think I can experiment with forcing the VP8 codec but no promises 😄

[Daki0082]

Okay, thanks!

[SpikeHD]

@Daki0082 Okay, after some investigation, this seems to be more of a "thing" than I thought, so if you're willing, I'd like to get you to answer/try a couple things for me:

1. What is your setup like? Are you using multi-monitors? Are you using 1 monitor connected to your motherboard, and one connected to your dedicated GPU? Do you even have a dedicated GPU?
2. Open the DevTools console in Dorion (Ctrl+Shift+i) and paste and run the following code (yea yea I know this is sketch, you don't have to if you don't want to). This will output your GPU details, I want to make sure whatever hardware you're running is getting detected properly since Dorion should use hardware acceleration. Just paste the output back to me:

   
   var canvas = document.createElement('canvas');
   var gl;
   var debugInfo;
   var vendor;
   var renderer;

try { gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl'); } catch (e) {}

if (gl) { debugInfo = gl.getExtension('WEBGL_debug_renderer_info'); vendor = gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL); renderer = gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL); }

console.log(vendor) console.log(renderer)

Mine outputted:

Google Inc. (NVIDIA) VM692:20 ANGLE (NVIDIA, NVIDIA GeForce RTX 3070 (0x00002484) Direct3D11 vs_5_0 ps_5_0, D3D11)

3. If you have a friend willing to put in some time (NOT just using an alt locally, that won't give accurate data. trust me, I've tried, lol), I'd like you to look at the following:
  a. You, in **Dorion**, start a screenshare like normal, ideally of a game or something you know tanks the FPS
  b. Your friend, in a normal client, opens the stream. Then, they click the green "Voice Connected" text, and on the bottom of that popup, they should see "Debug". Click that. If it doesn't show up, [enable debug mode](https://www.howtogeek.com/714348/how-to-enable-or-disable-developer-mode-on-discord/) in Discord and try again.
   ![image](https://github.com/SpikeHD/Dorion/assets/25207995/c8a85a9a-ad65-471b-af38-55bdb464d9d0)
  c. In the box that should now appear, they click on the user (you) that is streaming (under "RTC Debug: Media"), which should open something like below. Make sure it is the "Video" debug tab, not "Audio". **Screenshot that and send it here as well**, below is what you're looking for:![image](https://github.com/SpikeHD/Dorion/assets/25207995/b0a75c58-3146-4e39-8dca-848509ed214c)

I will also hopefully remember to at least add an option for disabling hardware acceleration in the next release which may or may not help, if I am unable to fix this before the next one.

For any nerds reading, my thoughts:
1. Since WebRTC is P2P, it is interesting that when reproducing locally (Dorion streaming and browser viewing, on the same PC), this problem seems to not exist. This suggests it could be a network-related problem?
2. Screensharing also loves to crank the CPU like soulja boy. I'm noticing in Task Manager that screensharing uses GPU 0 (Intel HD). Unsure why that's the case when Dorion detects my GPU just fine, but it may be because I use a weird setup that uses the iGPU for one monitor, and my 3070 for the other?

[SpikeHD]

Tested with a friend as well, my findings. Firefox was my browser:

1. Browser screensharing good. Closer to 30fps. Bitrate was fine.
2. Dorion screensharing consistently bad. Never faltered from low FPS (5-ish). Bitrate was fine.
3. Browser webcam consistently good. Close to target FPS. Bitrate and resolution matched target.
4. Dorion webcam started bad, then gradually improved. Started at low FPS, went up to close to target. Bitrate started terrible as well, ended about the same as browser.

All-in-all, I think something is up with WebView2s implementation of screensharing, as webcam seemed to work fine. Going to look into this further but you are still welcome to answer the stuff above.

EDIT: tested with Chrome (as that is what WebView2, AKA MSEdge, runs under the hood), results:

1. Webcam -> Exact same as Dorion, started bad and made its way to good
2. Screenshare -> Only marginally better, if at all. Max FPS also low, bitrate was fine

This at LEAST means I can Google "low fps chrome" instead of "low fps webview2", which should give me better results, lol

[SpikeHD]

@Daki0082 Another discovery: streaming a full desktop instead of a window will make the framerate totally fine. You can also share audio when streaming entire desktops instead of Windows. Give that a try, see if that is the case for you as well.

[SpikeHD]

Created an upstream issue in WebView2Feedback: https://github.com/MicrosoftEdge/WebView2Feedback/issues/4176

[Daki0082]

@Daki0082 Okay, after some investigation, this seems to be more of a "thing" than I thought, so if you're willing, I'd like to get you to answer/try a couple things for me:

1. What is your setup like? Are you using multi-monitors? Are you using 1 monitor connected to your motherboard, and one connected to your dedicated GPU? Do you even have a dedicated GPU?
2. Open the DevTools console in Dorion (Ctrl+Shift+i) and paste and run the following code (yea yea I know this is sketch, you don't have to if you don't want to). This will output your GPU details, I want to make sure whatever hardware you're running is getting detected properly since Dorion should use hardware acceleration. Just paste the output back to me:

var canvas = document.createElement('canvas');
var gl;
var debugInfo;
var vendor;
var renderer;

try {
  gl = canvas.getContext('webgl') || canvas.getContext('experimental-webgl');
} catch (e) {}

if (gl) {
  debugInfo = gl.getExtension('WEBGL_debug_renderer_info');
  vendor = gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL);
  renderer = gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL);
}

console.log(vendor)
console.log(renderer)

Mine outputted:

Google Inc. (NVIDIA)
VM692:20 ANGLE (NVIDIA, NVIDIA GeForce RTX 3070 (0x00002484) Direct3D11 vs_5_0 ps_5_0, D3D11)

This is what mine outputted:

Google Inc. (NVIDIA)
ANGLE (NVIDIA, NVIDIA GeForce GTX 1650 (0x00001F0A) Direct3D11 vs_5_0 ps_5_0, D3D11)

I'm going to ask a friend of mine to help me soon. I will respond again with the screenshot when I have it

[Daki0082]

Here's the screenshot of the debugging thing:

[SpikeHD]

Okay, thank you, it seems it is what I feared most: a bug with the underlying browser engine :(

I can't fix this myself, but I did submit a (hopefully) detailed issue in WebView2Feedback, so hopefully that leads to something. In the meantime, you can try screensharing your entire desktop instead of specific windows, since I'm pretty sure the bug is present in the window capturing system. At least, sharing my desktop works fine for me, feel free to report back if it doesn't for you 😄

[cozykitten]

Fullscreen sharing with audio, same problem, friend also said it looked like 5fps

Google Inc. (NVIDIA)
VM447:18 ANGLE (NVIDIA, NVIDIA GeForce GTX 1060 6GB Direct3D11 vs_5_0 ps_5_0, D3D11)

[SpikeHD]

Yep, nothing I can do until the upstream issue gets an update!

[noideas4thisname]

Yep, definitely encountered this. Makes Dorion unusable for this use case at the moment. Unfortunately upstream seems to not have any traction that we can see.

[SpikeHD]

I've experimented with the Vencord webScreenShareFixes plugin, and it seemed to bring the bitrate from ~2500 Kbps to ~3500 Kbps but in more complex scenarios (I tested with Overwatch practice range) it still has very poor FPS.