KwaiVGI / LivePortrait

Bring portraits to life!
https://liveportrait.github.io
Other
12.7k stars 1.35k forks source link

Can more keypoints increase driving scale? long hair cannot be driven correctly #29

Open jcn16 opened 4 months ago

cleardusk commented 3 months ago

More keypoints maybe better, but its hard to specify the best number of keypoints. Honestly, non-diffusion based methods are hard to simulate the hair swaying.

molegod commented 3 months ago

@cleardusk Is there anyway to specify keypoints in the current implementtaion?

jcn16 commented 3 months ago

Thanks for your reply. I visulized the implicit keypoints and found all keypoints are close to face region. I wonder whether driving scale can be enlarged by adding more fluffy hair or long hair data.

jcn16 commented 3 months ago

@cleardusk Is there anyway to specify keypoints in the current implementtaion?

I guess you can specify keypoints by "Landmark-guided implicit keypoints optimization" mentioned in paper.

oneThousand1000 commented 2 months ago

Thanks for your reply. I visulized the implicit keypoints and found all keypoints are close to face region. I wonder whether driving scale can be enlarged by adding more fluffy hair or long hair data.

Hi, how did you visualize the implicit keypoints? Did you do some 3D-to-2D things? I am confused because there seems to be no projection matrix in the project.

=== update ===

Finally, I tried to visualize all implicit points and found that they seem to have no explicit semantic information. I also tried to change those points with 3D keypoints (estimated by 3DMM), the results were bad and unrealistic. I think the model should be re-trained to achieve explicit control.