Configurable finger photo gesture

[uyjulian]

Is your feature request related to a problem? Please describe.

On Valve Index Knuckles controllers, it is difficult to fit the conditions for taking the finger photo on the first time, or unintentionally taking a finger photo.

Even if the 3 fingers are touching the grip area, the fingers may not appear all down in-game, in which case turning off and turning back on the controller may allow you to try it again.

It is much easier to do this on Oculus Touch controllers since there is only a single grip button which controls the position of those three fingers in-game.

Describe the solution you'd like

Instead of reading the skeleton from the steam vr runtime, read the controller data directly.

E.g. if a certain percentage like 75 percent of the grip surface area is touched, then consider it as 3 fingers down for the purpose of the finger photo conditions. If the index finger is touching the trigger (instead of just hovering over it), consider it as taking the finger photo.

Describe alternatives you've considered

Replacing the skeleton might be an alternative choice

Additional Context

n/a

[Beethoven-n]

one thing i found that helps is using your lasers to align for the pose you need to do

[FlameSoulis]

one thing i found that helps is using your lasers to align for the pose you need to do

Doesn't dismiss how much of a pain it is to do on an Index. Plus, some avatars have misleading directions for the fingers/laser.

[Beethoven-n]

yeah you're right. it should be easier

[troyBORG]

Yeah there has been times I have to do the finger trill to reprogram the controllers to get my fingers all down. Or sometimes because the touch sensor on my left stick is dead my thumb will be down so I don't get the photo until I touch the stick to get it to reset its position.

I almost would like a thing where you can say "this is the gesture I want to use".

Like how on your phone you have to train your fingerprint unlock, have it you do the gesture you want multiple times so it learns how you pose your hands for the finger photo to trigger.

[shiftyscales]

For some additional context- the only finger that is required to be held down is the middle finger. You will also need to keep your thumb off of the touchpad/thumbstick/face buttons as it positions the thumb in a way that disables the gesture.

As others noted- the underlying issue is in the Index Controllers not reliably detecting finger curl- there is a built-in mechanism to calibrate the controllers by drumming your fingers along the controller.

If the gesture is ever giving me a hard time, I do that really quickly to ensure the fingers are tracking properly/the middle finger is properly lowered, and that I'm not touching any of the face buttons and the gesture works fairly reliably for me.

I agree it could be improved- and more importantly- this functionality could be made accessible as a native feature for other controller types rather than being coincidental (Oculus Touch controllers are technically not intended to work at all with the finger photo functionality as it was built with finger tracking in mind specifically.)

In future I could see some value in there being a calibration process in which the user performs their desired gesture multiple times in a row, rather than being a hardcoded gesture- but for now- I hope the above tips helped.

[shiftyscales]

It's also worth considering that a single gesture might not be perfectly suitable for all users- making this functionality something that can be configured/calibrated could do a better job of ensuring users can perform this gesture if they are not able to do so with the default pose for any given reason, e.g. they don't have full control over both arms, or are otherwise restricted in their mobility.

[Frooxius]

This is a bit tricky, because the gesture is a bit of a sliding scale. Making it easier to trigger also often increases the accidental takes - in fact, it's been made harder to trigger, because people would do it accidentally too many times.

The gesture is based purely on the skeleton though, which is intentional - it makes it work both with controllers and with finger tracking solutions. We are unlikely to move away from this design and implement different solutions for different controllers, because this would require us to design several differing solutions, instead of one unified one.

However the actual logic of the gesture should be reworked a bit, which should help improve how it behaves overall. There's a lot of "wrong" ways to trigger it, which shouldn't be possible, so there's something that is messed up with the logic on triggering the gesture, but I haven't had time to really look into this.

[TisFoolish]

What about instead of moving away from the design, allowing users to define a custom gesture / hand pose per avatar on top of fixing the issues with it falsely triggering? I feel like that would fulfill the need of OP while also adding accessibility because not all users can do the default way of doing finger photos.

[Frooxius]

That would require moving away from the design as well and make this even more complex task.

Instead of a system that's fine tuned for this gesture, we would need to implement a robust system to define custom gestures, which is at least order of magnitude more complex engineering-wise.

[TisFoolish]

I've been lead to believe that the finger photo system has an internal set of rotations for the hands and fingers that it compares against and if the the hands and fingers are "close enough" it enables the finger photo, however it seems that may not be the case? It'd be good to know how it actually determines when to turn on and off, it'd at least help in figuring out why some avatars have incredibly hard to trigger finger photo.

[shiftyscales]

It's not avatar specific at all, @TisFoolish. It's equally possible to do the gesture on an avatar with no hands.

But my best guess would be if a user is using a visual reference for how their avatar's hands are posed in-app- and their avatar's hands are not perfectly aligned to that in the same way, that could account for some amount of difference.

The rotation of the hand proxies are set up at the time of the avatar's creation when placing the hand visuals, and can later be adjusted in the IK settings.

What matters is where the user's controllers are positioned, and the state of the fingers (in specific as noted above- the middle finger being fully curled, and the thumb being at approximately the corect angle (which Steam's skeletal input likes to move based on the orientation of the hand, and is also why touching the face buttons, etc. can disrupt the gesture.)

[troyBORG]

Yeah it's mostly positioning on your actual hands. Like I can use any model. Doing finger photos with the NeoFur models is possible.

What controllers are you using, I can actually trigger it even easier now on the quest pro controllers vs the index controllers.

[shadowpanther]

From my experience:

• Oculus controllers via Oculus runtime (no SteamVR) - good repeatable pose, because finger poses are quite fixed (thumb is either up or down, for example)
• Knuckles on SteamVR - a bit tricky, because finger poses are a continuum, and the controller often recalibrates the touch values/weights
• Quest controllers via SteamLink on SteamVR - can't do the gesture, because for fixed finger poses SteamVR adds a curl depending on hand orientation

[FlameSoulis]

* Quest controllers via SteamLink on SteamVR - can't do the gesture, because for fixed finger poses SteamVR adds a curl depending on hand orientation

I can confirm this with the Quest Pro via SteamLink. In fact, when I was having trouble, the people in the session suggested turning the right hand by 180 degrees, as that was "the solution to the problem."

The community knows this feature is very problematic and even frustrating to use due to its fickle nature, which is a shame because it is a neat idea.