met4citizen / TalkingHead

Talking Head (3D): A JavaScript class for real-time lip-sync using Ready Player Me full-body 3D avatars.
MIT License
296 stars 95 forks source link

Attach prop? #28

Closed JPhilipp closed 5 months ago

JPhilipp commented 5 months ago

Is there a way to parent a 3D object to the avatar? I'm not even looking to have it look like it's held in the hand or anything, just something roughly before the person (ideally sticking to the avatar pivot). I got the glb objects ready.

And then also, a way to remove that prop again, or swap it out with another one. Perhaps all with the ability to set the relative position.

Background: I'm thinking of letting players add story-influencing props, like "Staff of Bad Luck", to their adventure story host.

Cheers!

met4citizen commented 5 months ago

Here is a rough outline of how you can load and add a GLB object half a meter away from the avatar's head and then remove it:

import * as THREE from 'three';
import { GLTFLoader } from 'three/addons/loaders/GLTFLoader.js';
...
const loader = new GLTFLoader();
let gltf = await loader.loadAsync( "props.glb" );
let props = gltf.scene.getObjectByName("Props");
head.scene.getObjectByName("Head").add(props);
props.position.set(0.5,0,0);
...
head.scene.getObjectByName("Head").remove(props);
head.clearThree(props);

There are, of course, many ways to do this and manipulate the object. If you not familiar with the three.js library, you can find the documentation here.

JPhilipp commented 5 months ago

Thanks, it works now!

For reference and in case anyone comes across similar issues, here's the code I ended up with -- I did some automatic meshes collecting (independent of the meshes names contained with the glb I add), adding it to a group I can position and handle at once, while ensuring I clone the objects during scene traversal so as to not cause an error with the scene. Passing an empty modelId will clear the object.

let storyPropGroup;

export async function avatarSetStoryProp(modelId) {
  if (!head) { return; }

  console.log("avatarSetStoryProp", modelId);

  if (storyPropGroup) {
    await head.scene.getObjectByName("Head").remove(storyPropGroup);
    await head.clearThree(storyPropGroup);
    storyPropGroup = null;
  }

  if (modelId) {
    const url = `/story-props/${modelId}.glb`;
    console.log("url", url);

    const loader = new GLTFLoader();
    let gltf = await loader.loadAsync(url);

    storyPropGroup = new THREE.Group();

    gltf.scene.traverse(function (object) {
      if (object.isMesh) {
        let clone = object.clone();
        storyPropGroup.add(clone);
      }
    });   

    head.scene.getObjectByName("Head").add(storyPropGroup);

    storyPropGroup.position.set(0.5, 0, 0);
    storyPropGroup.scale.set(0.7, 0.7, 0.7);
  }
}