search

TimoBolkart / voca

This codebase demonstrates how to synthesize realistic 3D character animations given an arbitrary speech signal and a static character mesh.
https://voca.is.tue.mpg.de/en
1.14k stars 271 forks source link

Is the the vertice's coordinates in registered data the same as it in data_verts.npy in training data? #128

Open aurelianocyp opened 2 months ago

aurelianocyp commented 2 months ago

I found the first frame in sentence01 of FaceTalk_170725_00137_TA in registereddata is: [[ 0.04883447 0.01008725 -0.01469336] [ 0.05236208 0.00897936 -0.01322775] [ 0.05311589 0.01053391 -0.01269362] ... [-0.05066473 0.04737104 0.05660217] [-0.04855442 0.04771237 0.05538856] [-0.04625392 0.04809992 0.05461453]]

however , the same frame in data_verts is: [[ 0.06330613 -0.0130609 -0.05464683] [ 0.06681567 -0.01411122 -0.05312932] [ 0.0675151 -0.0125118 -0.05253972] ... [-0.03784862 0.02131454 0.01536897] [-0.03572937 0.02173329 0.01420005] [-0.03342807 0.0221916 0.01347607]]

TimoBolkart commented 1 month ago

The data used for training are unposed, meaning that effects of global translation, global rotation, and the head rotation around the neck are removed. Please refer to Section 6 of the VOCA paper for details about this data pre-processing.