AdaRoseCannon / handy-work

Framework Agnostic Hand tracking for WebXR
https://aframe-xr-starterkit.glitch.me/
MIT License
94 stars 11 forks source link

Fix pose detection after suspend/session restart and right hand detection #3

Closed msub2 closed 3 years ago

msub2 commented 3 years ago

I've gotten the A-Frame scene to work now for the left hand through multiple session starts, stops, and suspends (on Quest 2). Will look into why the right hand doesn't seem to be getting picked up next.

msub2 commented 3 years ago

Alright, looks like it's working as intended now! Short summary of changes:

msub2 commented 3 years ago

After more testing I've found that handDataToFile doesn't appear to be working in the A-Frame example, as it's expecting both hands to be present in inputSources but it's only ever getting called with either the left or right one. Will see if there's an elegant way to fix that.

msub2 commented 3 years ago

Alright, what I ended up doing was making __dumpHands a window value (since it seemed like you were referring to it as such on one line but not the others) and moved the hands object from handDataToFile outside the function. This way when the left hand calls it first it gets added to the object, then the right hand calls it and proceeds with the pose dumping before clearing the tempHands object.

I also added three new poses for testing; the sign of the horns, the shaka sign, and the Vulcan salute. Interestingly enough with these three new poses I'm starting to see some strange overlaps (most of what used to be left hand 'relax' now seems to be getting read as 'shaka', etc). Probably something to do with how it's detecting the poses/what the original ones were? I don't quite grasp exactly how that works yet, but hopefully this can help with testing and refining it.

AdaRoseCannon commented 3 years ago

This is amazing, thank you so much for doing this!

AdaRoseCannon commented 3 years ago

I included a weights array in the schema so that you can make some fingers less or more important. I.e. Thumbs are not important in many cases. Making a weights editor for rapid tweaking may take a while though.

AdaRoseCannon commented 3 years ago

We should still expose resetHands, it seems handy (pardon the pun), but I would rather not use it in the AFrame code, since the bug isn't framework specific.

Can we put the calls for it into handy-work.js, that way it will be fixed no matter what framework is used also that way we have fewer objects attached to window

msub2 commented 3 years ago

Sure thing, will take a look at it later today.

msub2 commented 3 years ago

Should be good to go now!

AdaRoseCannon commented 3 years ago

Thank you so much! I'll take a look now.

AdaRoseCannon commented 3 years ago

Thinking about it since various session events cause the hands to be reset perhaps it maybe best to tell the user when to trigger reset rather than doing it ourselves.

AdaRoseCannon commented 3 years ago

Merging, but will change the interface slightly before putting into main