zhanglonghao1992 / One-Shot_Free-View_Neural_Talking_Head_Synthesis

Pytorch implementation of paper "One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing"
Other
735 stars 141 forks source link

Head angle rotation question #48

Open oo92 opened 2 years ago

oo92 commented 2 years ago

Hi.

If I feed in selfie_1.png of a person looking in direction_A and selfie_2.png of a person looking in direction_b, how can I use your model to have selfie_1.png look in direction_B while preserving identity (facial hair, hair style, piercings, etc)?

zhanglonghao1992 commented 2 years ago

For an image rather than a video, especially the face orientation or posture is very different, it is recommended to use the face swapping algorithm. :)

oo92 commented 2 years ago

@zhanglonghao1992 Faceswapping algorithm does not work for beards, especially prominent ones.

zhanglonghao1992 commented 2 years ago

@oo92 It can't preserve selfie_1's beard or selfie_2's beard?

zhanglonghao1992 commented 2 years ago

@oo92 If it can't keep 1's beard, it means that your supervision of the identity is not good enough (probably because your identity encoder is not robust?)

oo92 commented 2 years ago

Say that selfie_1 (with beard) is looking straight and selfie_2 is looking to the left. I don't want to do a face swap. I want to move selfie_1's head to the same direction as selfie_2, thereby making them look at the exact same angle. When doing this, I want to make sure I keep everything about selfie_1 intact, including the beard.

oo92 commented 2 years ago

Not a face swap but think of a headswap

zhanglonghao1992 commented 2 years ago

@oo92 Then take selfie_1 as the source and selfie_2 as the driving, and slightly modify the code of the demo.py.

oo92 commented 2 years ago

@zhanglonghao1992 Modify which part of the file?