hand-tracking-controls-extras

Disclaimer

When I rushed at a-frames finger tracking I wasn't aware, that I'm kinda reinventing the wheel. A great place with multiple examples is Marlon Lückerts repository.

Another great repo with gesture detection is Ada Rose Cannon handy-work

On the other hand, I've learned a lot doing this, and am fully willing to continue :) To all starring, and forking people - thank You! Any ideas are more then welcome ( i.e. in form of an issue )

Navigation components

Teleport, drag yourself around, or rotate the world like this:

https://user-images.githubusercontent.com/17348360/155236391-98df435e-2949-4ea1-8a72-b7ca017ada9a.mp4

With a simple setup like this:

<a-entity id="rig">
  <a-camera></a-camera>
  <!-- left hand can teleport, and drag the world position -->
  <a-entity hand-tracking-controls="hand: left" hand-tracking-extras
  hand-teleport="rig: #rig; origin: a-camera" drag-move="rig: #rig; speed: 5">
  </a-entity>

  <!-- right hand can rotate the world by dragging -->
  <a-entity hand-tracking-controls="hand: right" hand-tracking-extras drag-rotate="rig: #rig">
  </a-entity>
</a-entity>

More info in the docs

Simplified (?) Joints API

Hands in browser:

Import the script:

<script src="https://gftruj.github.io/hand.tracking.controls.extras/dist/aframe-hand-tracking-controls-extras.js"></script>

Add the hand-tracking-extras to the hand:

<a-entity id="left-hand" hand-tracking-controls="hand: left; " hand-tracking-extras></a-entity>

Wait for the hand-tracking-extras-ready event:

var hand = document.getElementById("left-hand");
hand.addEventListener("hand-tracking-extras-ready", (evt) => {
  var jointsAPI = evt.detail.data.jointsAPI;
});

Or grab the joints manually:

var joints = document.getElementById("left-hand").components["hand-tracking-extras"].jointsAPI
var Wrist = joints.getWrist();
console.log(Wrist.getPosition());

Check out the API and the XRHand docs.

API

Each joint has its "helper" method:

Arguments are optional, but providing them will save memory (as otherwise internal helpers are cloned and returned);

Joints

Grab the joints with:

// jointsAPI.get<Finger><Bone>()
// example: const index_tip = jointAPI.getIndexTip();
// 
// Fingers: Wrist(special case, single bone), Index, Middle, Ring, Little
// Bones: Metacarpal, Proximal, Intermediate, Distal, Tip

From the WebXR hand docs:

Similar to the WebXR hand API, the joints are

Roadmap - wishful thinking

Provide:

1. a simple API for Joints orientation and relations:

   • .islookingUp(), isHorizontallyAligned(), .isCloseTo(other), orientedLike(other)

2. Integrate navigation components with Adas handy-work