Fine tuning to optimize a specific face output

[EylonSho]

Thank you so much for publishing this great implementation!

I'd like to use the pre-trained model in order to drive a specific face to talk, by another driving person's talking head.

• I don't mind to overfit on this desired specific person in the output.
• Both the target person and the driver one are not included in the VoxCeleb dataset (so the pre-trained model generates a 3D photo-realistic output, but it does not similar enough to the actual face).

How would you suggest to fine tune the model? Whether should I train with the target face talking? Or whether with the driving person? Or both of them?

Any comments would be very appreciated! 😄

[zhanglonghao1992]

@EylonSho I haven't tried fine tune the model for a specific person yet. But I think it's hard to succeed unless you have a large number of pictures of this person. For a specific person, maybe you should try some NeRF methods.

[EylonSho]

Thank you the tip! I'll try that.