what is the difference between the target nerual pose and the target nerual deformed pose?

vasiliskatr / deformation_transfer_ARkit_blendshapes

Implementation of the deformation transfer paper and its application in generating all the ARkit facial blend shapes for any 3D face

MIT License

63 stars 10 forks source link

what is the difference between the target nerual pose and the target nerual deformed pose? #1

Closed ForrestPi closed 2 years ago

ForrestPi commented 2 years ago

I am a new learner about CG. I am confued about the difference between Neutral.obj and ictinner2ARkit.obj Could you explain it and how to get the mesh of nerual deformed pose

Thanks!

vasiliskatr commented 2 years ago

Hi ForrestPi, In order to transfer the deformations from one topology to the other we first need to find good correspondences for each vertex between the target mesh and source mesh.

The neutral.obj is the target face (i.e. target mesh) for which we want to create the blend shapes.
The ictinner2ARkit.obj is the Neutral.obj which has been deformed using NRICP so that it matches the shape of the neutral face from the generic ARkit blend shapes (source mesh).

Now we can use this new, deformed version of the target mesh (ictinner2ARkit.obj) to find good correspondences with the source mesh (ARkit neutral face) and perform deformation transfer for each of the blend shape expressions as we only need to find good vertex correspondences between the meshes once!

I hope this helps!

ForrestPi commented 2 years ago

Thanks your patient reply. "The ictinner2ARkit.obj is the Neutral.obj which has been deformed using NRICP " the vertexs of neutral.obj(target mesh) is (3, 6706) the vertexs of neutral.obj(arkit mesh) is (3,1220) I try to viz Neutral.obj ictinner2ARkit.obj and Neutral.obj(arkit) using meshlab

I found that they are not aligned could you share your nricp code how to match the shape of the neutral face from the generic ARkit blend shapes (source mesh).

vasiliskatr commented 2 years ago

The Meshes do not need to be aligned initially.

Rigid alignment takes place in the initial blocks of dt.ipynb please refer to it and the comments there.

As I mention in the readme of this repo. I use the commercial software Wrap3D to perform NRICP between target (ict topology) and source (ARkit topology) neutral faces and yield the ictinner2ARkit.obj which is then used in the code for finding vertex correspondences between the two topologies.

ForrestPi commented 2 years ago

Thanks, I got it!