search

amirbar / speech2gesture

code for training the models from the paper "Learning Individual Styles of Conversational Gestures"
361 stars 41 forks source link

Misalignment of audio and frames #16

Open ShenhanQian opened 3 years ago

ShenhanQian commented 3 years ago

According to the file common/consts.py, we know that

SR = 16000
AUDIO_SHAPE = 67267
FPS = 15
FRAMES_PER_SAMPLE = 64

From the first three constants, we can compute num_frames = AUDIO_SHAPE / SR * FPS = 67267 / 16000 * 15 = 63.06281249999999 which is about one whole frame less than FRAMES_PER_SAMPLE.

We have encountered this problem when we were trying to test the model on a longer audio sequence, for which the misalignment is magnified.