stalgiag / p5.xr

a library that helps turn p5.js sketches into immersive experiences using WebXR
GNU Lesser General Public License v2.1
117 stars 25 forks source link

Feat/hand tracking #217

Open TiborUdvari opened 3 months ago

TiborUdvari commented 3 months ago

Not quite ready to merge this, but to follow up on the API proposal in #216 it would look something like this

https://github.com/stalgiag/p5.xr/assets/1434442/3c4356b1-56d6-44aa-a587-525e81f50d63

function preload() {
  createARCanvas();
}

function setup() {
  describe("A sphere on your right index finger");
  mainHandMode(RIGHT);
}

function draw() {
  normalMaterial();
  push();
  translate(finger.x, finger.y, finger.z);
  sphere(0.01);
  pop();
}
TiborUdvari commented 3 months ago

https://github.com/stalgiag/p5.xr/assets/1434442/dcff6863-a1a4-4148-a599-69712fa628b0

This would allow to have quick interactions like this in an "active area", a spatial equivalent to

function draw() {
  ellipse(mouseX, mouseY, 10, 10);
}
TiborUdvari commented 3 months ago

Full hands

https://github.com/stalgiag/p5.xr/assets/1434442/3992a974-c488-4fc6-859c-e7e503d9a09d

function preload() {
  createARCanvas();
}

function setup() {
  describe("Spheres on all the skeletal joints of the hand.");
}

function draw() {
  normalMaterial();
  for (let i = 0; i < hands.length; i++) {
    const joint = hands[i];
    push();
    translate(joint.x, joint.y, joint.z);
    sphere(0.01);
    pop();
  }
}
TiborUdvari commented 3 months ago

Other experiments here: https://www.tiborudvari.com/sketchbook

TiborUdvari commented 3 months ago

Also worth mentioning this depends on the #215 refactor

stalgiag commented 1 month ago

Amazing!