search

zhangchenxu528 / FACIAL

FACIAL: Synthesizing Dynamic Talking Face With Implicit Attribute Learning. ICCV, 2021.
GNU Affero General Public License v3.0
376 stars 83 forks source link

valid loss of audio2face #69

Open stupiding opened 2 years ago

stupiding commented 2 years ago

I noticed that the finetuning of audio2face model just use all the data as train set. When split the data into train and valid, I found the loss of generator overfitted. And another worth-noting thing is, the variation of audio2face's prediction is much smaller than ground truth. So is there anything we can do to alleviate the overfit, or to enlarge prediction's variantion?

zhangchenxu528 commented 2 years ago

Good questions. For the overfit, you may decrease the learning rate or use the early stop.

The variation will be reduced if the length of the finetuning video is too long. You may increase the step in dataset102. Now, it is set to 1. (list(np.arange(0, min_num_frame-self.frames, 1) + base))