search

pygfx / wgpu-py

The next generation GPU API for Python
https://wgpu-py.readthedocs.io
BSD 2-Clause "Simplified" License
391 stars 31 forks source link

Pinch and other Gestures #509

Open hmaarrfk opened 1 month ago

hmaarrfk commented 1 month ago

Describe the bug

Some of my users complained about "not being able to zoom". We couldn't figure it out until one of them tried to do so on my laptop, and we noticed they were trying to pinch on the trackpad to zoom.

Are these kinds of gestured captured in the WGPU/PyGFX stack? I tried to search of "pinch" in the codebase and couldn't find anything.

To Reproduce

Truthfully, its a little difficult for me to test this since I am linux and "pinch" is really only "supported" by "wayland" and xwayland is used to build the wgpu canvas.

Observed behavior

Users accustomed to apple's zoom in "gestures" try to pinch to zoom with 2 fingers on the trackpad with the PanZoomController on PyGFX and they say "well that doesn't work" and don't zoom into the images.

Your environment

 $ mamba list | grep -E "(wgpu|pyside|pygfx)"
pygfx                     0.2.0              pyhd8ed1ab_0    conda-forge
pyside6                   6.7.0           py310h7a3fa89_1    conda-forge
wgpu-native               0.19.3.1             h2b8f863_0    conda-forge
wgpu-py                   0.15.2             pyha804496_0    conda-forge
██ system:

             platform:  Linux-6.8.0-31-generic-x86_64-with-glibc2.39
python_implementation:  CPython
               python:  3.10.13

██ versions:

wgpu:  0.15.1

Generally running Wayland + PySide6

hmaarrfk commented 1 month ago

(probably not a bug, but a feature request or just general brainstorming)

Korijn commented 1 month ago

I believe our canvases emit pointer events and each event includes a pointer id. This id would be different for each finger. With that you should be able to implement pinch gesture (and others). Therefore I would think this is already supported sufficiently at the level of wgpu-py.

If your expectation is for pygfx controllers to support this, you would have to update those to handle multi pointer gesture events.

hmaarrfk commented 1 month ago

hmm interesting, I tried it on OSX and Wayland, and the event.pointer_id always reports 0.

to be clear I mean "pinch to zoom on the touchpad"

https://support.apple.com/en-us/102482 image

ps. no "expectations" for me, just trying to understand how one would go about to see if I can make it a pygfx wide change.

almarklein commented 1 month ago

Two-finger up-down scrolling seems to work though. Pinching does not seem to generate any events. This code is helpful to test this sort of thing: https://github.com/pygfx/wgpu-py/blob/main/examples/events.py).

almarklein commented 1 month ago

GLFW does not have pinch support, it seems https://github.com/glfw/glfw/issues/90

For Qt seems to be viable to add support for pinch events. I'm having a stab.

Korijn commented 1 month ago

I'm going to guess that it does already work (even on mac and wayland) in jupyter rfb. It is going to be different for each combination of GUI backend and OS

almarklein commented 1 month ago

I put what I have now in #515. Seems doable, although support will be limited to certain gui backends. And needs work in pygfx to support these events. Note that in addition to scaling, the pinch gesture also has rotation, which can be useful to e.g. manipulate objects in the scene.