search

Weizhi-Zhong / IP_LAP

CVPR2023 talking face implementation for Identity-Preserving Talking Face Generation With Landmark and Appearance Priors
Apache License 2.0
637 stars 72 forks source link

the lip shakes heavily after training #33

Open MYTHOFLEGEND opened 11 months ago

MYTHOFLEGEND commented 11 months ago

Hello, I trained the landmarks model by the videos I collected from bilibili, running_L1_loss is 0.0066 and running_velocity_loss is 0.0048. but in the inference result, the lip shakes heavily and mismatch with the upper face

https://github.com/Weizhi-Zhong/IP_LAP/assets/48516484/bddc213a-cc13-41bf-b349-c602e4888a7a

could you give me some suggestions to fix that? Thanks very much!

1059692261 commented 11 months ago

同样存在这个问题，不过我安装的一些库可能跟作者在requirements列的不太一样，不知道是不是这个原因。请问你是按照作者给的requirements安装的环境吗？

MYTHOFLEGEND commented 11 months ago

不是，我都是尽量装的最新，pytorch用的2.0.1，直接用作者的模型推理就没事，自己训练的就会出现这个问题

Samkulin commented 10 months ago

我用预训练模型都会抖

komilaria commented 10 months ago

@MYTHOFLEGEND hello bro!! how to get separate video or separate video for show_sketch and separate video for full

komilaria commented 10 months ago

do you have any ideas to improve resolution?

qwert1887 commented 7 months ago

@MYTHOFLEGEND Hi! If possiable, Can you tell me what you used to remove the background in your video.I'm very interested in this.Thanks in advance.

sunjian2015 commented 5 months ago

上下对不上可能是因为推理和训练时 crop 人脸时设置的 pad 不一致导致的，这个代码里处理数据时是 5，推理时是 25