How to avoid distortion of facial and hand features in generated videos

fudan-generative-vision / champ

Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance

https://fudan-generative-vision.github.io/champ/

MIT License

4.72k stars 597 forks source link

How to avoid distortion of facial and hand features in generated videos #116

Open CodeConductor520 opened 5 months ago

CodeConductor520 commented 5 months ago

When I continued testing with the official provided model, I found that using the official provided images as reference images resulted in better facial and hand effects in the generated video; But when using our own images, the facial and hand distortions are more severe, is it because the official reference images have been trained? Will training our own model improve facial and hand distortion issues? Looking forward to your reply.

XinZhang0526 commented 5 months ago

Hi! I have same problem with you. Do you follow the SMPL & Rendering doc to produce your own motion datas? I guess it's the key. I'm trying to do so. if work, I will share here! Keep contact!

gabrielevccp commented 4 months ago

@XinZhang0526 did you see any improvements?

AricGamma commented 4 months ago

This is a known issue and we are trying new ways to solve it

sahir2k commented 3 months ago

changing the scheduler's beta_schedule to "scaled_linear" makes it more stable . also , you can increase the smoothing intensity in the smoothing blender file ...

https://github.com/fudan-generative-vision/champ/assets/117151647/5cbd3081-9b27-43ce-8f0a-fb5aa04924b4