search

foivospar / NED

PyTorch implementation for NED (CVPR 2022). It can be used to manipulate the facial emotions of actors in videos based on emotion labels or reference styles.
https://openaccess.thecvf.com/content/CVPR2022/html/Papantoniou_Neural_Emotion_Director_Speech-Preserving_Semantic_Control_of_Facial_Expressions_in_CVPR_2022_paper.html
MIT License
155 stars 26 forks source link

Can the method be tested directly on images (not videos)? #3

Open TianxiangMa opened 1 year ago

TianxiangMa commented 1 year ago

Can the method be tested directly on images (not videos)? For example, two images are used to test expression transfer.

foivospar commented 1 year ago

Hi,

Unfortunately, no. The method is designed for videos, so the Emotion Manipulator expects a sequence of length >= 10. The only implicit way to make it work for images would be to create a short video by copying the image 10 times. Still, you would have to train the neural face renderer (assuming you test on a different celebrity than the ones provided), which, given only 1 image, would not generalize well.