mouse pointer issue with sommelier

halt-spesn commented 2 years ago

mouse pointer aren't capture in 3d fps games, so looking in game is impossible, any fix?

supechicken commented 2 years ago

Try enabling the #exo-pointer-lock (or something name similar) flag in chrome://flags

halt-spesn commented 2 years ago

Try enabling the #exo-pointer-lock (or something name similar) flag in chrome://flags

in v108 this flag is gone, in v105 this flag exist but not working

halt-spesn commented 2 years ago

UPD: sommelier pointer capture works in crostini vm, but not in crouton, i think because sommelier commit from 2018, maybe it can be fixed?

myelin commented 1 year ago

@shifty-d I'll check in with the other Borealis folks to see if anyone has any suggestions. I've confirmed that pointer lock doesn't appear to be working in my latest Sommelier build in Crouton, built from src/platform2/vm_tools/sommelier with https://crrev.com/c/4021320.

Reproduction steps:

Install bullseye with my crouton fork using the instructions at https://github.com/myelin/crouton.
Build Sommelier; my command line (a bit hacky because I'm rsyncing the build tree from my Google workstation, which probably already has any required virtwl headers included)

cd ~/crouton_sommelier; sudo chown -R $USER:$USER .; rm -rf build; meson build; meson configure build -Dxwayland_path=/usr/bin/Xwayland; meson configure build -Dxwayland_gl_driver_path=; (cd build; ninja) && XDG_RUNTIME_DIR=/var/run/chrome ~/crouton_sommelier/build/sommelier --noop-driver --force-drm-device=/dev/dri/renderD128 --glamor -X glxinfo -B

Run Steam under Sommelier:

XDG_RUNTIME_DIR=/var/run/chrome ~/crouton_sommelier/build/sommelier --noop-driver --force-drm-device=/dev/dri/renderD128 --glamor -X steam -no-cef-sandbox

Play Tomb Raider. Observe that the mouse is stuck inside a rectangle, so you can't pan forever.

myelin commented 1 year ago

Poking around in the Chrome source:

https://crsrc.org/c/components/exo/wayland/zwp_pointer_constraints.cc handles the Wayland message; when it gets a pointer lock request, it calls pointer->ConstrainPointer, which is in https://crsrc.org/c/components/exo/pointer.cc.

I'm guessing that Crouton windows don't have the right security delegate (see the security_delegate->CanLockPointer call in pointer.cc), so the pointer lock request is being ignored. If you run Sommelier with WAYLAND_DEBUG=1, you'll probably see the zwp_locked_pointer request failing.

It looks like Crostini and Borealis both have their own SecurityDelegate overrides that enable this, and otherwise Exo has its DefaultSecurityDelegate which allows ARCVM and Lacros windows to lock the pointer.

So I'm afraid this is unlikely to work properly without help from Chrome.

I managed to hack my way around it by doing this:

Run Borealis (although Crostini would probably work too).
Find the secure Exo socket folder (see notes in https://crsrc.org/c/components/exo/security_delegate.h) under /run/wayland -- in my case it was /run/wayland/concierge/scoped_dirKDfTIB.
Bind-mount that into Crouton:

mkdir /var/run/crouton/media/removable/extssd/crouton/chroots/test-bullseye/borealis-wayland
mount --bind /run/wayland /var/run/crouton/media/removable/extssd/crouton/chroots/test-bullseye/borealis-wayland

Run Steam under Sommelier, using Borealis's Wayland socket instead of the default one:

XDG_RUNTIME_DIR=/borealis-wayland/concierge/scoped_dirKDfTIB/ ~/crouton_sommelier/build/sommelier --noop-driver --force-drm-device=/dev/dri/renderD128 --glamor -X steam -no-cef-sandbox

Run Tomb Raider, and verify that pointer lock works -- I can spin the camera around for as long as I like without hitting a pointer wall.

So this works, although requires you to launch Crostini or Borealis as well as Crouton. It's possible that ARCVM or Lacros might also create a wayland-0 socket with pointer lock privileges too, although I'm less familiar with either of them.

halt-spesn commented 1 year ago

Poking around in the Chrome source:

https://crsrc.org/c/components/exo/wayland/zwp_pointer_constraints.cc handles the Wayland message; when it gets a pointer lock request, it calls pointer->ConstrainPointer, which is in https://crsrc.org/c/components/exo/pointer.cc.

I'm guessing that Crouton windows don't have the right security delegate (see the security_delegate->CanLockPointer call in pointer.cc), so the pointer lock request is being ignored. If you run Sommelier with WAYLAND_DEBUG=1, you'll probably see the zwp_locked_pointer request failing.

It looks like Crostini and Borealis both have their own SecurityDelegate overrides that enable this, and otherwise Exo has its DefaultSecurityDelegate which allows ARCVM and Lacros windows to lock the pointer.

So I'm afraid this is unlikely to work properly without help from Chrome.

I managed to hack my way around it by doing this:

Run Borealis (although Crostini would probably work too).

Find the secure Exo socket folder (see notes in https://crsrc.org/c/components/exo/security_delegate.h) under /run/wayland -- in my case it was /run/wayland/concierge/scoped_dirKDfTIB.

Bind-mount that into Crouton:
mkdir /var/run/crouton/media/removable/extssd/crouton/chroots/test-bullseye/borealis-wayland
mount --bind /run/wayland /var/run/crouton/media/removable/extssd/crouton/chroots/test-bullseye/borealis-wayland
Run Steam under Sommelier, using Borealis's Wayland socket instead of the default one:

XDG_RUNTIME_DIR=/borealis-wayland/concierge/scoped_dirKDfTIB/ ~/crouton_sommelier/build/sommelier --noop-driver --force-drm-device=/dev/dri/renderD128 --glamor -X steam -no-cef-sandbox

Run Tomb Raider, and verify that pointer lock works -- I can spin the camera around for as long as I like without hitting a pointer wall.

So this works, although requires you to launch Crostini or Borealis as well as Crouton. It's possible that ARCVM or Lacros might also create a wayland-0 socket with pointer lock privileges too, although I'm less familiar with either of them.

Lacros that's intresting, can you say more about lacros?

myelin commented 1 year ago

I’m afraid I’m not very familiar with Lacros. It’s Linux Chrome on ChromeOS, and apparently has its own Wayland socket that supports pointer lock, according to the comments in the code I linked above.

halt-spesn commented 1 year ago

I’m afraid I’m not very familiar with Lacros. It’s Linux Chrome on ChromeOS, and apparently has its own Wayland socket that supports pointer lock, according to the comments in the code I linked above.

i checked, lacros doesn't have dedicated sockets

dnschneid / crouton

mouse pointer issue with sommelier #4814