AdaRoseCannon / handy-work

Framework Agnostic Hand tracking for WebXR
https://aframe-xr-starterkit.glitch.me/
MIT License
94 stars 11 forks source link

AFrame: Don't render any controllers or hands in AR #8

Open AdaRoseCannon opened 2 years ago

AdaRoseCannon commented 2 years ago

Don't render controllers or hands in AR since the user can see the physical hardware/wetware anyway

diarmidmackenzie commented 1 year ago

Ideally in AR you'd render the hands using a "hider material" so that they occlude any rendered objects they are in front of. Otherwise distant rendered objects mask out the hands, which breaks immersion.

Without modifying these components, it's possible to implement this using 2 small additional components like this:

AFRAME.registerComponent('hider-material', {

  dependencies: ['material'],

  init() {
    const material = this.el.components.material.material
    material.colorWrite = false
  }
})

AFRAME.registerComponent('handy-render-order', {

  dependencies: ['handy-controls'],

  init() {

    this.el.addEventListener('model-loaded', () => {

      const hands = ['hand-mesh-left', 'hand-mesh-right']
      hands.forEach((hand) => {
        const model = this.el.getObject3D(hand)
        if (model) {
          model.renderOrder = -1
        }
      })
    })
  }
})

and then in the HTML:

    <a-scene renderer="sortObjects: true">

and

<a-entity handy-controls="materialOverride: both" material hider-material handy-render-order>

Might be nice to roll this capability into handy-work as a native feature at some point.

Works pretty well, except that the hand models seem a bit too small (for my hands, at least), so the occlusion isn't quite perfect.

I'm not yet sure why the hand meshes are coming out on the small side, but probably best tracked as a seoarate issue.