Larger face angles and facial features positioning

[ChuixiongWu]

I tried using LivePortrait for facial expression cloning and I was unbelievable how well it worked. However, I still noticed some issues where the simulated face has some noticeable up-and-down zoom when the face is turned at a large angle, especially when the driving video is tilted up, which can lead to distortion. Does LivePortrait have a location for the facial features? Because I used to have positioning problems with DeepFaceLab sometimes, I used Machine Movie Editor to modify anchor points. I wonder if LivePortrait uses a similar positioning technology? Can I open up the relevant interfaces so that I can adjust the anchors myself? I really need to do some big-angle facial imitations

[ChuixiongWu]

Is there any information about frames and face anchors in the temporary .pkl file?

[ChuixiongWu]

I checked the face key points obtained by my driver video through the insightface-bafalo_l model and found that there was no problem. The main reason for this issue is that the first frame does not stay centered, plus there is a lot of shaky in the recorded video. I used the Comfy UI to do this with the newly added nodes. If anyone has a need, I can make it public.

[legolaswang]

I checked the face key points obtained by my driver video through the insightface-bafalo_l model and found that there was no problem. The main reason for this issue is that the first frame does not stay centered, plus there is a lot of shaky in the recorded video. I used the Comfy UI to do this with the newly added nodes. If anyone has a need, I can make it public.

你好，能否分享一下你具体是怎么做的？

[ChuixiongWu]

你好，能否分享一下你具体是怎么做的？

我使用的是LivePortrait的Comfy UI版本，那里面会使用insightface和bafalo_l模型获得人脸的关键点，以此作为LivePortrait模型的输入。 我在大角度的driving video情况下进行人脸替换，效果不好。一开始，我怀疑关键点是不准确的，使得我的LivePortrait不能很好的工作，最终导致了合成效果不理想。 因此我修改了节点的一部分代码，使得它可以有一个额外的输出，即带有关键点点的人脸图像。我查看了这些关键点，发现定位的非常准确。因此排除了这个可能性。 随后我又重新录制了一段driving video. 这次我更加严格，尽可能让人脸居中。事实证明这样效果好了很多。 总的来说，LivePortrait对driving video有比较高的要求，需要初始帧人脸居中并正面向前，随后要尽可能使人脸居中。