emersion / xdg-desktop-portal-wlr

xdg-desktop-portal backend for wlroots
MIT License
598 stars 61 forks source link

Getting Zoom to screen share #319

Open DhruvaSambrani opened 2 months ago

DhruvaSambrani commented 2 months ago

Hi, I use dwl which is built on wlroots, and would like to screen share on Zoom 6.2.0 (after Zoom said they fixed screen share on wayland). When trying to screen share, I get the share dialog from zoom as expected, but selecting "System Share" or the other doesn't do anything, not even share a black screen. It is clear screen share is not active from UI or behaviour.

I have checked the wiki and made sure that XDG_CURRENT_DESKTOP=wlroots for dbus, systemd and terminal shell. zoomus.conf has xwayland=false, and enableWaylandShare=true. slurp is installed, and gUM check works well. The one time gamescope was running (now gamescope doesn't run due to unrelated issue), zoom was able to share, but shared only a black screen, probably because nothing else was running in gamescope? I tried starting Zoom with XDG_CURRENT_DESKTOP=GNOME, to no avail.

Does anyone have a clue as to why it doesn't work?

mylesbartlett72 commented 2 months ago

zoom and wayland screen share is rather messy in my experience.

my preferred method to work around this is to use obs to capture the screen and output it using v4l2loopback (load this before starting obs) and then share a second camera from zoom (note that you may have to make sure to select your primary camera for video first, or stick it on top of the screen share in obs before opening zoom).

you can also just use the virtual camera as your primary camera, but zoom will blur it since it doesnt treat it as a screen share and uses an unsuitable lossy compression algorithm. second camera shares fix this (you can share your primary camera by first disabling your video in zoom and then initiating a share)

its quite messy but it worked well enough for me back when I regularly used zoom. unfortunately, i think the screen sharing issues are a general problem with zoom on wayland.

smlx commented 2 months ago

I assume you are using the Zoom Electron app? As an alternative the Zoom PWA works well for screen share in Sway.

DhruvaSambrani commented 2 months ago

you can also just use the virtual camera as your primary camera

yes this is what I do now, but still would prefer to use it normally.

As an alternative the Zoom PWA works well

Yes but it has some other issues

David96 commented 1 month ago

I took a quick look at DBUS logs and apparently Zoom just gives up if it doesn't find the RemoteDesktop protocol... (I guess it wants it for the remote control feature?). So adding #263 to my installation, Zoom now at least manages to bring up the screen selector. After choosing a screen though it crashes. From DBUS it seems it never calls the ScreenCast Start method, only the RemoteDesktop.Start, maybe it gets confused there? The sequence looks like this:

method call time=1727701365.757097 sender=:1.95 -> destination=:1.94 serial=183 path=/org/freedesktop/portal/desktop; interface=org.freedesktop.impl.portal.RemoteDesktop; member=Start
   object path "/org/freedesktop/portal/desktop/request/1_81/zoomcast7"
   object path "/org/freedesktop/portal/desktop/session/1_81/zoomcast4"
   string ""
   string ""
   array [
   ]
method return time=1727701365.757138 sender=:1.95 -> destination=:1.81 serial=184 reply_serial=52
   object path "/org/freedesktop/portal/desktop/request/1_81/zoomcast7"
method return time=1727701365.757427 sender=:1.94 -> destination=:1.95 serial=23 reply_serial=183
   uint32 0
   array [
      dict entry(
         string "devices"
         variant             uint32 3
      )
   ]
error time=1727701365.757488 sender=:1.94 -> destination=:1.95 error_name=org.freedesktop.DBus.Error.UnknownMethod reply_serial=183
   string "Unknown method Start or interface org.freedesktop.impl.portal.RemoteDesktop."

Which seems weird to me since there's a reply to the "Start" followed by an error that the method doesn't exist.

Anyways, I don't know much about dbus and Zoom just seems to have a policy of "make it work on one computer and ignore any error paths" instead of correctly implementing any protocols. It's very annoying and time consuming to try to figure out where they went wrong again so let's hope they fix it at some point?

DhruvaSambrani commented 1 month ago

I have no idea if this will help, but running zoom through Gamescope "allows" me to successfully screen share. However, since gamescope doesn't have any other windows, it just displays a black screen, but I think it works otherwise. Since gamescope is OS, maybe we can see what is different between our and their implementations of the protocol?

jkyang92 commented 1 month ago

I'm not certain if this will help, but looking through the zoom logs (~/.zoom/logs/zoom_stdout_stderr.log), I have the following output:

Linux Client Version is 6.2.3 (2056)
QSG_RENDER_LOOP is 
XDG_CURRENT_DESKTOP = sway;   GDMSESSION = ;   XDG_SESSION_TYPE = wayland
/usr/libexec/xdg-desktop-portal-wlr: unrecognized option '--version'
Usage: xdg-desktop-portal-wlr [options]

    -l, --loglevel=<loglevel>        Select log level (default is ERROR).
                                     QUIET, ERROR, WARN, INFO, DEBUG, TRACE
    -c, --config=<config file>        Select config file.
                                     (default is $XDG_CONFIG_HOME/xdg-desktop-portal-wlr/config)
    -r, --replace                    Replace a running instance.
    -h, --help                       Get help (this text).

[xdg-desktop-portal info]:
   xdg-desktop-portal: xdg-desktop-portal 1.18.4
   can not find 'xdg-desktop-portal-gnome' command
   xdg-desktop-portal-gtk: xdg-desktop-portal-gtk 1.14.1
   can not find 'xdg-desktop-portal-hyprland' command
   can not find 'xdg-desktop-portal-kde' command
   can not find 'xdg-desktop-portal-lxqt' command
   xdg-desktop-portal-wlr: xdg-desktop-portal-gtk 1.14.1
   can not find 'xdg-desktop-portal-xapp' command
[portal info] :
   gnome-keyring.portal
   gtk.portal
   kwallet.portal
   wlr.portal
[pipewire info] :
   pipewire
   Compiled with libpipewire 1.2.5
   Linked with libpipewire 1.2.5

There's unrelated output before and after it, but at some point it seems something is running xdg-desktop-portal-wlr --version. I have no idea if this is causing an issue or just an unrelated issue. But other than that, I have the same symptoms as @DhruvaSambrani

I'm on sway on Gentoo. I have previously (earlier this year and on an older version) been able to screen share with zoom, but it doesn't seem to work anymore.

DhruvaSambrani commented 1 month ago

I'm on sway on Gentoo. I have previously (earlier this year and on an older version) been able to screen share with zoom, but it doesn't seem to work anymore.

This precisely. I can definitely remember screen sharing on Wayland (dwl), but don't remember how.

kmrozek-shareablee commented 1 month ago

Yup, the issue happens because Zoom started using RemoteDesktop protocol. There's no proper implementation of it in xdg-desktop-portal-wlr :(.

DhruvaSambrani commented 1 month ago

So #2 ?

mahkoh commented 1 month ago

From DBUS it seems it never calls the ScreenCast Start method, only the RemoteDesktop.Start, maybe it gets confused there?

zoom calls

all with the same session. This is supported and the reply to the start method must also create a screencast and send the stream in the reply as it would with a pure screencast session.

The clue is in the error log:

ERROR:../zoom/src/Meeting/As/Wayland/wlscreencast.cpp:300:void on_start_screencast_response_received_cb(GDBusConnection*, const char*, const char*, const char*, const char*, GVariant*, gpointer): assertion failed: (g_variant_iter_n_children (&iter) == 1)

and then looking at which start method was called and what part of the reply contains an empty array.


I've implemented this in my portal and confirmed that it works: https://github.com/mahkoh/jay/pull/291/commits/260d241f79c521493362e0c6a8c4344bf6fba69f

I only support version 2 of the RDP though. I don't know if it would also work with an implementation that only supports version 1.

David96 commented 1 month ago

Yes, looking at the protocol a bit closer I also realized that RemoteDesktop.start is supposed to start the screencast session as well. I added a (very hacky) patch to #263 which does exactly that and Zoom now succesfully starts screen sharing (though crashes when ending it, but that appears to be the case on every DE due to a pipewire update).

I wasn't aware of your compositor @mahkoh , but it seems like its portal is tightly integrated into the compositor and not using the wayland protocols? If it were, it would be a nice alternative to this portal implementation, that's why I'm checking.

The patch:

```diff diff --git a/include/screencast.h b/include/screencast.h index 99b7411..4fc18c1 100644 --- a/include/screencast.h +++ b/include/screencast.h @@ -6,4 +6,6 @@ void xdpw_screencast_instance_destroy(struct xdpw_screencast_instance *cast); void xdpw_screencast_instance_teardown(struct xdpw_screencast_instance *cast); +int start_screencast(struct xdpw_screencast_instance *cast); + #endif diff --git a/src/remotedesktop/remotedesktop.c b/src/remotedesktop/remotedesktop.c index 51f4ddb..0eac0bd 100644 --- a/src/remotedesktop/remotedesktop.c +++ b/src/remotedesktop/remotedesktop.c @@ -1,7 +1,10 @@ #include "remotedesktop.h" #include +#include +#include "screencast.h" +#include "screencast_common.h" #include "wlr_virtual_pointer.h" #include "xdpw.h" @@ -191,6 +194,25 @@ static int method_remotedesktop_start(sd_bus_message *msg, void *data, sd_bus_er return -1; } logprint(DEBUG, "remotedesktop: start: session found"); + struct xdpw_screencast_instance *cast = sess->screencast_data.screencast_instance; + logprint(DEBUG, "remotedesktop: screencast instance %x", cast); + + if (cast) { + logprint(DEBUG, "remotedesktop: starting screencast"); + if (!cast->initialized) { + ret = start_screencast(cast); + } + while (cast->node_id == SPA_ID_INVALID) { + int ret = pw_loop_iterate(state->pw_loop, 0); + if (ret < 0) { + logprint(ERROR, "pipewire_loop_iterate failed: %s", spa_strerror(ret)); + return ret; + } + } + } + if (ret < 0) { + return ret; + } remote = &sess->remotedesktop_data; remote->virtual_pointer = zwlr_virtual_pointer_manager_v1_create_virtual_pointer( @@ -229,11 +251,107 @@ static int method_remotedesktop_start(sd_bus_message *msg, void *data, sd_bus_er return ret; } - ret = sd_bus_reply_method_return(msg, "ua{sv}", PORTAL_RESPONSE_SUCCESS, - 1, "devices", "u", POINTER | KEYBOARD); + sd_bus_message *reply = NULL; + ret = sd_bus_message_new_method_return(msg, &reply); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_append(reply, "u", PORTAL_RESPONSE_SUCCESS); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_open_container(reply, 'a', "{sv}"); + if (ret < 0) { + return ret; + } + + ret = sd_bus_message_append(reply, "{sv}", + "devices", "u", POINTER | KEYBOARD); + if (ret < 0) { + return ret; + } + + ret = sd_bus_message_open_container(reply, 'e', "sv"); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_append(reply, "s", "streams"); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_open_container(reply, 'v', "a(ua{sv})"); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_open_container(reply, 'a', "(ua{sv})"); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_open_container(reply, 'r', "ua{sv}"); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_append(reply, "u", cast->node_id); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_open_container(reply, 'a', "{sv}"); + if (ret < 0) { + return ret; + } + if (cast->target->output->xdg_output) { + ret = sd_bus_message_append(reply, "{sv}", + "position", "(ii)", cast->target->output->x, cast->target->output->y); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_append(reply, "{sv}", + "size", "(ii)", cast->target->output->width, cast->target->output->height); + if (ret < 0) { + return ret; + } + } + ret = sd_bus_message_append(reply, "{sv}", "source_type", "u", MONITOR); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + + ret = sd_bus_message_append(reply, "{sv}", + "devices", "u", KEYBOARD | POINTER); + if (ret < 0) { + return ret; + } + + ret = sd_bus_message_close_container(reply); + if (ret < 0) { + return ret; + } + + ret = sd_bus_send(NULL, reply, NULL); if (ret < 0) { return ret; } + sd_bus_message_unref(reply); return 0; } diff --git a/src/screencast/screencast.c b/src/screencast/screencast.c index 43a7b80..889e70a 100644 --- a/src/screencast/screencast.c +++ b/src/screencast/screencast.c @@ -186,7 +186,7 @@ bool setup_target(struct xdpw_screencast_context *ctx, struct xdpw_session *sess } -static int start_screencast(struct xdpw_screencast_instance *cast) { +int start_screencast(struct xdpw_screencast_instance *cast) { int ret; ret = xdpw_wlr_session_init(cast); if (ret < 0) { ```
mahkoh commented 1 month ago

(though crashes when ending it, but that appears to be the case on every DE due to a pipewire update).

It doesn't crash on my system but I'm using the flatpak.

I wasn't aware of your compositor @mahkoh , but it seems like its portal is tightly integrated into the compositor and not using the wayland protocols?

Yes, I only posted the commit for reference.

David96 commented 1 month ago

The clue is in the error log:

ERROR:../zoom/src/Meeting/As/Wayland/wlscreencast.cpp:300:void on_start_screencast_response_received_cb(GDBusConnection*, const char*, const char*, const char*, const char*, GVariant*, gpointer): assertion failed: (g_variant_iter_n_children (&iter) == 1)

Where did you get that from by the way? Looks like it includes zoom debug symbols?! That would make things so much easier if they break things again!

mahkoh commented 1 month ago

~/.var/app/us.zoom.Zoom/.zoom/logs/zoom_stdout_stderr.log for the flatpak and ~/.zoom/logs/zoom_stdout_stderr.log otherwise.

David96 commented 1 month ago

Hmm, I'm aware of that file but it never included such detailed error messages for me. Maybe the version flatpak uses has some additional logging :thinking:

David96 commented 1 month ago

Thanks for all the discussions here, it’s helpful for us to improve. We will better improve screenshare under wayland in the future.

You're kind of implying you actually work for Zoom so just in case something I would really appreciate:

Don't fail silently! Yes, Linux is difficult because it is so diverse. Not every desktop supports every protocol. When using something like RemoteDesktop/Screencast, please check whether it is supported first (best not by using heuristics like XDG_CURRENT_DESKTOP but by actually querying dbus) and tell the user if it is not! Be specific in the message and mention the exact protocol, this way no one has to play a game of guessing what's missing and we can just implement the missing feature.

In this case, Zoom actually seems to have done most things right by using a protocol the way it is documented. It would be even better of course if Zoom wouldn't crash if the portal misbehaves.

That being said, I'm using Zoom now with my fork (https://github.com/David96/xdg-desktop-portal-wlr/commits/remotedesktop) and screencasting works. Still crashing when screen sharing is being stopped though.

kmrozek-shareablee commented 4 weeks ago

That's it works :). Can you create a PR? It would be great of other users can just install xdg-desktop-portal-wlr from repo with your changes. This one looks dead https://github.com/emersion/xdg-desktop-portal-wlr/pull/263 :).

David96 commented 4 weeks ago

In principle yes, but the RemoteDesktop protocol is much bigger than just adding this hack to make Zoom work. I don't have the time to spent a huge amount of effort here, maybe @columbarius or @emersion could have a look at my branch (https://github.com/David96/xdg-desktop-portal-wlr/commits/remotedesktop) and give a rough estimate of how much would have to be changed for it to get merged into main? Would e.g. not implementing touchscreen support be a reason to not merge it at all? Or the fact, that the keyboard code is fully copied from wayvnc?

emersion commented 2 weeks ago

I don't think any PR needs to be a complete RemoteDesktop impl before it can be merged. In fact, breaking the work into smaller chunks would make it easier to review.