search

alvr-org / alvr-visionos

Experimental visionOS client for ALVR - SteamVR on Apple Vision Pro!
MIT License
426 stars 30 forks source link

Acceleration data for using only hands/gestures? #109

Open maatjes2 opened 5 months ago

maatjes2 commented 5 months ago

I read somewhere that the AVP doesn't allow for transmitting the acceleration data from the Joycons, but would it be possible to send it when using only a hands/gestures setup? Something like calculating direction and differences in hand positions over time? It seems many people cannot play games like Half-Life: Alyx (which require acceleration data from the moment you pick up the "gravitation gloves") due to this data lacking.

Maybe some inspiration can be taken from the following fork of the original, which calculates linearVelocity and angularVelocity as derivative of position and rotation data: https://github.com/dnnkeeper/ALVR/tree/ControllerVelocityFix

shinyquagsire23 commented 4 months ago

Technically we do send this, it's just the streamer that's not actually using it atm

maatjes2 commented 3 months ago

Technically we do send this, it's just the streamer that's not actually using it atm

Seems the guys over at the streamer are pointing back to the client: https://github.com/alvr-org/ALVR/issues/2212

Do JoyCons send acceleration data (from a "throwing gesture" for example), and is this data passed on by the ALVR-visionos client to the streamer/server? @shinyquagsire23 Do you own a couple of JoyCons and would you be able to check whether you're able to throw objects in Half-Life: Alyx? Seems that the items are dropping straight (vertically) to the ground, as soon as the grip is released. I would like to know if this is a limitation due to the Joy-Cons? If so, it should be possible for the client to calculate force/acceleration based on Vision Pro hand-tracking instead, right? (calculating linearVelocity and angularVelocity as derivative of position and rotation data). Maybe people using Joy-Cons (as well as people using no controllers/hands only) should then be able to tick a box in the client that confirms they are using them (if this cannot be detected automatically), and the client should send acceleration data based on recent hand-tracking position history? Especially with the recent upgrade of hand-tracking to 90 fps, this should be accurate enough for playability.

zaptrem commented 2 months ago

Technically we do send this, it's just the streamer that's not actually using it atm

Seems sending acceleration data was left as a TODO? https://github.com/alvr-org/alvr-visionos/blob/986eed819753c75884ed5645f46d6444f8a7d5f7/ALVRClient/WorldTracker.swift#L970