search

sahilg06 / EmoGen

PyTorch Implementation for Paper "Emotionally Enhanced Talking Face Generation" (ICCVW'23 and ACM-MMW'23)
Creative Commons Zero v1.0 Universal
343 stars 27 forks source link

perceptual Loss and sync loss cannot be used at the same time during training #3

Closed dy0829 closed 1 year ago

dy0829 commented 1 year ago

When I train the Wav2Lip model, adding the sync expert loss on top of the perceptual loss causes the training to collapse. I noticed that you also use perceptual loss in your training process, and I'm wondering if you have encountered a similar situation. If so, how did you solve it?

sahilg06 commented 1 year ago

I didn't encounter such a situation during training. Also, I cannot think of a reason why perceptual loss and sync loss cannot be used simultaneously. However, I did encounter a situation where my training collapsed even before adding perceptual loss. I changed the optimizer's hyperparameters a bit for the same (here).

dy0829 commented 1 year ago

Thanks for your reply. I will check my code again.