Vencord / venmic

linux audio screenshare for discord (or any electron app) via pipewire
https://www.npmjs.com/package/@vencord/venmic
Mozilla Public License 2.0
86 stars 5 forks source link

[Requires Vesktop Plugin] Remove Discord Screenshare down-mixing #12

Closed Kyuunex closed 4 months ago

Kyuunex commented 11 months ago

✍️ Bug Description

Go Live Screenshare audio is mono

🔁 Steps to reproduce

Account 1: stream something with stereo audio Account 2: view account 1's stream with stereo headphones

🗒️ Debug Output

[2023-12-16 02:40:07.209] [venmic] [debug] [has_pipewire] pulse-server was pulseaudio (on pipewire 1.0.0)
[2023-12-16 02:40:07.214] [venmic] [info] found speakers: 52
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 28: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 29: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 37: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 52: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 59: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 61: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 72: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 82: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 128: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] ignoring 146: no props match
[2023-12-16 02:40:15.829] [venmic] [debug] trying to link 285, mic is 72
[2023-12-16 02:40:15.829] [venmic] [debug] target port(s): 2
[2023-12-16 02:40:15.829] [venmic] [debug] 305 has 1 corresponding port(s)
[2023-12-16 02:40:15.829] [venmic] [info] created link 151 (-> 285:FR)
[2023-12-16 02:40:15.829] [venmic] [debug] 100 has 1 corresponding port(s)
[2023-12-16 02:40:15.830] [venmic] [info] created link 114 (-> 285:FL)
[2023-12-16 02:40:15.830] [venmic] [debug] ignoring 311: no props match
[2023-12-16 02:40:15.918] [venmic] [debug] ignoring 310: no props match
[2023-12-16 02:40:15.919] [venmic] [debug] ignoring 310: no props match
[2023-12-16 02:40:15.919] [venmic] [debug] ignoring 310: no props match
[2023-12-16 02:40:15.919] [venmic] [debug] ignoring 310: no props match
[2023-12-16 02:40:21.160] [venmic] [debug] ignoring 308: no props match
[2023-12-16 02:40:21.164] [venmic] [debug] ignoring 103: no props match
[2023-12-16 02:40:21.216] [venmic] [debug] ignoring 197: no props match
[2023-12-16 02:40:21.216] [venmic] [debug] ignoring 197: no props match
[2023-12-16 02:40:21.225] [venmic] [debug] ignoring 291: no props match
[2023-12-16 02:40:21.225] [venmic] [debug] ignoring 291: no props match
[2023-12-16 02:40:35.928] [venmic] [debug] ignoring 93: no props match
[2023-12-16 02:40:35.928] [venmic] [debug] ignoring 93: no props match
[2023-12-16 02:41:00.707] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:41:00.707] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:41:05.739] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:05.739] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:06.370] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:06.370] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:06.379] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:06.379] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:06.379] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:06.379] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:08.549] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:08.549] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:09.503] [venmic] [debug] ignoring 191: no props match
[2023-12-16 02:41:09.503] [venmic] [debug] ignoring 191: no props match
[2023-12-16 02:41:09.503] [venmic] [debug] ignoring 191: no props match
[2023-12-16 02:41:09.503] [venmic] [debug] ignoring 191: no props match
[2023-12-16 02:41:09.504] [venmic] [debug] ignoring 122: no props match
[2023-12-16 02:41:09.504] [venmic] [debug] ignoring 122: no props match
[2023-12-16 02:41:09.504] [venmic] [debug] ignoring 122: no props match
[2023-12-16 02:41:09.504] [venmic] [debug] ignoring 122: no props match
[2023-12-16 02:41:09.506] [venmic] [debug] ignoring 68: no props match
[2023-12-16 02:41:09.506] [venmic] [debug] ignoring 68: no props match
[2023-12-16 02:41:09.506] [venmic] [debug] ignoring 68: no props match
[2023-12-16 02:41:09.506] [venmic] [debug] ignoring 68: no props match
[2023-12-16 02:41:09.527] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:09.527] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:09.527] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:09.527] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:09.528] [venmic] [debug] ignoring 277: no props match
[2023-12-16 02:41:09.528] [venmic] [debug] ignoring 277: no props match
[2023-12-16 02:41:09.528] [venmic] [debug] ignoring 277: no props match
[2023-12-16 02:41:09.528] [venmic] [debug] ignoring 277: no props match
[2023-12-16 02:41:09.529] [venmic] [debug] ignoring 86: no props match
[2023-12-16 02:41:09.529] [venmic] [debug] ignoring 86: no props match
[2023-12-16 02:41:09.529] [venmic] [debug] ignoring 86: no props match
[2023-12-16 02:41:09.529] [venmic] [debug] ignoring 86: no props match
[2023-12-16 02:41:09.530] [venmic] [debug] ignoring 96: no props match
[2023-12-16 02:41:09.530] [venmic] [debug] ignoring 96: no props match
[2023-12-16 02:41:09.530] [venmic] [debug] ignoring 96: no props match
[2023-12-16 02:41:09.530] [venmic] [debug] ignoring 96: no props match
[2023-12-16 02:41:09.531] [venmic] [debug] ignoring 116: no props match
[2023-12-16 02:41:09.531] [venmic] [debug] ignoring 116: no props match
[2023-12-16 02:41:09.531] [venmic] [debug] ignoring 116: no props match
[2023-12-16 02:41:09.531] [venmic] [debug] ignoring 116: no props match
[2023-12-16 02:41:09.531] [venmic] [debug] ignoring 143: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 143: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 143: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 143: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 89: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 89: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 89: no props match
[2023-12-16 02:41:09.532] [venmic] [debug] ignoring 89: no props match
[2023-12-16 02:41:12.592] [venmic] [debug] ignoring 194: no props match
[2023-12-16 02:41:12.592] [venmic] [debug] ignoring 194: no props match
[2023-12-16 02:41:12.612] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:12.612] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:12.612] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:12.612] [venmic] [debug] ignoring 107: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 302: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 302: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 59: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 59: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 59: no props match
[2023-12-16 02:41:15.285] [venmic] [debug] ignoring 59: no props match
[2023-12-16 02:41:15.294] [venmic] [debug] ignoring 288: no props match
[2023-12-16 02:41:15.294] [venmic] [debug] ignoring 288: no props match
[2023-12-16 02:41:15.294] [venmic] [debug] ignoring 288: no props match
[2023-12-16 02:41:15.294] [venmic] [debug] ignoring 288: no props match
[2023-12-16 02:41:15.296] [venmic] [debug] ignoring 126: no props match
[2023-12-16 02:41:15.296] [venmic] [debug] ignoring 126: no props match
[2023-12-16 02:41:15.296] [venmic] [debug] ignoring 126: no props match
[2023-12-16 02:41:15.296] [venmic] [debug] ignoring 126: no props match
[2023-12-16 02:41:15.899] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:15.899] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:15.899] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:15.899] [venmic] [debug] ignoring 141: no props match
[2023-12-16 02:41:15.912] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:15.912] [venmic] [debug] ignoring 41: no props match
[2023-12-16 02:41:15.926] [venmic] [debug] ignoring 180: no props match
[2023-12-16 02:41:15.926] [venmic] [debug] ignoring 180: no props match
[2023-12-16 02:41:15.926] [venmic] [debug] ignoring 180: no props match
[2023-12-16 02:41:15.926] [venmic] [debug] ignoring 180: no props match
[2023-12-16 02:41:23.739] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:23.739] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:23.739] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:41:23.739] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:42:28.436] [venmic] [debug] ignoring 173: no props match
[2023-12-16 02:42:28.437] [venmic] [debug] ignoring 173: no props match
[2023-12-16 02:42:28.448] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:42:28.448] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:42:28.448] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:42:28.448] [venmic] [debug] ignoring 186: no props match
[2023-12-16 02:42:28.614] [venmic] [debug] ignoring 108: no props match
[2023-12-16 02:42:28.615] [venmic] [debug] ignoring 108: no props match
[2023-12-16 02:42:28.626] [venmic] [debug] ignoring 177: no props match
[2023-12-16 02:42:28.626] [venmic] [debug] ignoring 177: no props match
[2023-12-16 02:42:28.627] [venmic] [debug] ignoring 177: no props match
[2023-12-16 02:42:28.627] [venmic] [debug] ignoring 177: no props match
[2023-12-16 02:42:37.309] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:42:37.309] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:42:37.309] [venmic] [debug] ignoring 179: no props match
[2023-12-16 02:42:37.309] [venmic] [debug] ignoring 179: no props match

========== STDOUT ==========

[~]$ VENMIC_ENABLE_LOG=1 vencord-desktop 
Gtk-Message: 02:45:15.039: Failed to load module "colorreload-gtk-module"
[87224:1216/024515.049981:ERROR:object_proxy.cc(577)] Failed to call method: org.freedesktop.DBus.Properties.Get: object_path= /org/freedesktop/portal/desktop: org.freedesktop.DBus.Error.InvalidArgs: No such interface “org.freedesktop.portal.FileChooser”
[87224:1216/024515.050001:ERROR:select_file_dialog_linux_portal.cc(280)] Failed to read portal version property
[arRPC > ipc] not available, trying again (attempt 1)
[arRPC > ipc] not available, trying again (attempt 2)
[arRPC > ipc] listening at /run/user/1000/discord-ipc-2
[arRPC > websocket] 6463 in use!
[arRPC > websocket] 6464 in use!
[arRPC > websocket] listening on 6465
[arRPC > process] started
[87260:1216/024516.537700:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 1 times!
[87260:1216/024516.543858:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 2 times!
[87260:1216/024521.708258:ERROR:gl_surface_presentation_helper.cc(260)] GetVSyncParametersIfAvailable() failed for 3 times!
'loop->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:426 pw_thread_loop_wait()
[2023-12-16 02:46:30.194] [venmic] [info] found speakers: 52
[2023-12-16 02:46:34.770] [venmic] [info] created link 267 (-> 285:FR)
[2023-12-16 02:46:34.771] [venmic] [info] created link 296 (-> 285:FL)
'loop->recurse > 0' failed at ../pipewire/src/pipewire/thread-loop.c:426 pw_thread_loop_wait()
[87260:1216/024942.349940:ERROR:shared_image_manager.cc(220)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.
[87260:1216/024942.351483:ERROR:shared_image_manager.cc(220)] SharedImageManager::ProduceSkia: Trying to Produce a Skia representation from a non-existent mailbox.

❔ Expected behavior

Stereo Audio

👀 Screenshots

2023-12-16T02:48:32,676503285+04:00

📦 Vesktop Version

Vencord 2cd8294 (Vesktop V0.4.4)

🖥️ Distribution

Arch (Hyprland)

✅ I confirm that...

Curve commented 11 months ago

As far as I can see this seems to either be a discord or vesktop limitation as venmic forwards the stereo audio

Kyuunex commented 11 months ago

perhaps there is a way around it?

Curve commented 11 months ago

perhaps there is a way around it?

Not from my side at least, depends on how discord screenshare is implemented, I vaguely remember rumors about audio always being merged into mono on discords side though

Kyuunex commented 11 months ago

I do know that microphone encoding (at one point) happened on the client side, and someone once wrote a BD plugin that changed the encoding options to make it stereo. Perhaps something similar could be done. If not, then it is what it is.

Curve commented 11 months ago

I do know that microphone encoding (at one point) happened on the client side, and someone once wrote a BD plugin that changed the encoding options to make it stereo. Perhaps something similar could be done. If not, then it is what it is.

Interesting idea, maybe you can open a plugin suggestion in the vencord (or rather vesktop as this is more geared towards vesktop specifically) repository with this issue as a reference.

It would may also help if you link the aforementioned plugin ^^

Kyuunex commented 11 months ago

im too lazy to make issues, so feel free to move this one around as you fit

here is the plugin i found on my hdd, i don't think it works anymore

//META{"name":"stereoSound"}*//

var stereoSound = function () {

  let VoiceConnection = BDV2.WebpackModules.findByUniqueProperties(['getVoiceEngine']).getVoiceEngine().VoiceConnection;

  class Stereo extends VoiceConnection {
    constructor(a, b, c, d, e) {
      super(a, b, c, d, e);
      this.origin = super.setTransportOptions;
    }
    setTransportOptions(obj) {
      if (obj.audioEncoder) {
        obj.audioEncoder.params = { stereo: "2" };
        obj.audioEncoder.channels = 2;
      }
      if (obj.fec) {
        obj.fec = false;
      }
      if (obj.encodingVoiceBitRate < 960000) {
        obj.encodingVoiceBitRate = 398000;
      }
      this.origin(obj);
      window.sound = this;
    }
  }

  return class _ {
    getName() { return "Stereo" }
    getDescription() { return "" }
    getAuthor() { return "" }
    getVersion() { return "1" }
    load() { }
    start() {
      BDV2.WebpackModules.findByUniqueProperties(['getVoiceEngine']).getVoiceEngine().VoiceConnection = Stereo;
    }
    stop() {
      BDV2.WebpackModules.findByUniqueProperties(['getVoiceEngine']).getVoiceEngine().VoiceConnection = VoiceConnection;
    }
  };
}();
Curve commented 11 months ago

I could reproduce this, however I supply discord with a stereo audio stream, it seems that discord somehow mixes them somewhere into mono - I'll close this for now, however this might be solvable with a vencord plugin

Curve commented 11 months ago

Vee just noted some things that we could try out to possibly make this work, will test those suggestions later

Vendicated commented 11 months ago

so i investigated and: the stream by venmic is correctly stereo, and it is also sent as stereo audio to discord. however, it seems like Discord downmixes it to mono. so it might be a server side restriction (note that vesktop uses WebRTC, while Discord Desktop uses UDP. the WebRTC api might just not support stereo)

we will investigate this further

Curve commented 4 months ago

Closing in favor of https://github.com/Vencord/Vesktop/issues/709