How could i get apple arkit 52 blendshapes?

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.

https://voca.is.tue.mpg.de/en

1.14k stars 271 forks source link

How could i get apple arkit 52 blendshapes? #97

Closed Justpenz233 closed 1 year ago

Justpenz233 commented 2 years ago

How could i get apple ARkit 52 blendshapes from the VOCA net-work output ? Should I train another net to do the mapping (from 5093 to 52) ?

TimoBolkart commented 1 year ago

One could output a lot of meshes from Apple's ARkit with different expressions, fit FLAME to these, and learn a mapping between FLAME's parameter space and the ARkit blendshapes.