search

OpenTalker / DPE

[CVPR 2023] DPE: Disentanglement of Pose and Expression for General Video Portrait Editing
https://carlyx.github.io/DPE/
MIT License
413 stars 44 forks source link

face deformation #18

Open zhewei-mt opened 10 months ago

zhewei-mt commented 10 months ago

Hello， Thank for your great work. I tried your code to perform pose transfer with the code python run_demo.py --s_path video.mp4 --d_path stable3.mp4 --model_path .\checkpoints\dpe.pt --face pose But the result is quite weired (see attachment). Is there any setting to improve the result?

https://github.com/OpenTalker/DPE/assets/109195411/1fc0ee68-bcf0-4450-a78a-24c7a49d03d6 video is down for same reason. Please see https://www.bilibili.com/video/BV1844y1F7v1/?vd_source=68fd0a3864408b733915dd2c8b2676f7.

Carlyx commented 10 months ago

Hi, Thank you for your attention! Due to the lack of structural information, rotating in the third dimension will generate artifacts. In fact, we have found that the version based on Transformer can solve this problem and is currently in progress. In addition, it may perform poorly on examples with large domain differences because the training set is Voxdataset.