search

OpenTalker / DPE

[CVPR 2023] DPE: Disentanglement of Pose and Expression for General Video Portrait Editing
https://carlyx.github.io/DPE/
MIT License
417 stars 45 forks source link

training code issue #22

Open tanshuai0219 opened 6 months ago

tanshuai0219 commented 6 months ago

Sorry, I didn't see the Expression loss (eq 10 in the paper) implemented in the training codes.

oriel commented 5 months ago

Hello, I have also the same observation about the missing expression recognition loss in the training code. I have tried to finetune the model on some new data, and it seems to not train correctly for better expression transfer, probably because of this missing loss. Would it be please possible to update the training code with the expression loss? Thanks a lot

xiaonae commented 3 weeks ago

Hello, I have also the same observation about the missing expression recognition loss in the training code. I have tried to finetune the model on some new data, and it seems to not train correctly for better expression transfer, probably because of this missing loss. Would it be please possible to update the training code with the expression loss? Thanks a lot

Hi, I have added the expression loss, you can find it in this link, http://openaccess.thecvf.com/content/CVPR2022/html/Danecek_EMOCA_Emotion_Driven_Monocular_Face_Capture_and_Animation_CVPR_2022_paper.html.

Could you share how you finetune the trained DPE on new data? Especially " the selection of loss function " and "the param.requires_grad of which submodules are set to True, including enc, dec_pose, dec_exp, mlp, mlp_exp, mlp_pose, and discriminator"?

Thank you very much!