search

ccloli / ALVR

Stream VR games from your PC to your headset via Wi-Fi
https://alvr-org.github.io/
MIT License
0 stars 0 forks source link

Emulating button press by hand pinching is not working #1

Open ccloli opened 1 year ago

ccloli commented 1 year ago

Hi @Meister1593 , sorry for bothering you.

I'm trying implementing button press from hand pinch, based on your draft PR (original in ALVR PR #1715), for self using and POC.

Since this is not a feature implemented by original ALVR itself, so I don't open the issue there.

image

From the runtime log, hand pinching is mostly detectable, and the button id is the same as the native button, but the button press doesn't trigger in SteamVR. In SteamVR controller binding ui, only the default joystick touch and trigger click are triggered, which is implemented by ALVR natively, and the new hand pinching emulated button press is not triggered.

I tried cherry-picking these changes to v20.1.0, too, but it's still not working. Since v20 is totally rewrited from v19, and the implementation of hand tracking is different (looks like in v20 it merges Oculus hand skeletal into OpenXR ways?), so I can't test them in v19.

The branch feature/hand-pinch-actions-poc is what I tried, and the changes can be seen from here. Not sure what am I missing, since I've little knowledge about Rust and never wrote it before. Maybe it's because the emulation is not in the thread where ALVR receives client packets? Or the emulation should be implemented in client side?

If you want to test it directly without local compile, you can try the prebuilt binary here. Though some of the releases may fail on some specific build task, most of them would have a Windows streamer executable binary.

Meister1593 commented 1 year ago

Hello I do know that hand tracking pinches don't work (but is detected), it seems like you need to do more than just change opts for them, i just haven't looked into pr actively (due to other ongoing issues, like automatic audio through pipewire on linux) V20 is completely different and mostly follows openxr specs (through pinching and what is available in there), and since not all vendors implement pinching yet (or other actions), we are trying to emulate it.

Meister1593 commented 1 year ago

Besides this, also hysteresis has to be implemented (thresholds for pinch activation/deactivation), but i wanted to do that after i dealt with button presses

ccloli commented 1 year ago

I do know that hand tracking pinches don't work (but is detected), it seems like you need to do more than just change opts for them

Yep I could guess I need to do more than that, like adding threshold to prevent sending multiple press in a short time (seems that SteamVR doesn't like it and may ignore the action, that's why I save the previous pressing state and compare), but other than that I've no idea. Do you have any thought? Though from my little experience I don't think I could fix it.

Meister1593 commented 1 year ago

I was as confused as you are, so i'm not sure what's up with that, maybe @zarik5 could shine some light onto this

ccloli commented 1 year ago

That's sad, anyway thanks for your kindly help and explanation. 😉

ccloli commented 1 year ago

I think cdbfa4631918749c7d4492ef48c6ffc575e52bb2 fixes the issue, by merging my changes with this commit, the buttons can be emulated as pressed now. Though in SteamVR the controller binding preview doesn't highlight the pressed button, the button is actually pressed on in-game controller (also sends to game).

https://github.com/ccloli/ALVR/assets/8115912/5201cc42-6e75-4b9e-88a1-5d68298b4d8d

But the default trigger click is gone somehow. 🤣

BTW another PR https://github.com/alvr-org/ALVR/pull/1794 is addressing the same feature, maybe we can follow it.