search

ChenFengYe / motion-latent-diffusion

[CVPR 2023] Executing your Commands via Motion Diffusion in Latent Space, a fast and high-quality motion diffusion model
https://chenxin.tech/mld/
MIT License
586 stars 55 forks source link

Question about generating result jitter #29

Open cyk990422 opened 1 year ago

cyk990422 commented 1 year ago

Hi, thank you so much for sharing such an interesting job. May I ask if the results generated by your VAE in the early stage of training, such as 1000-2000 epochs, will have motion jitter, and do you need to add some smoothing filters such as euros?

ChenFengYe commented 1 year ago

Hi, thanks for your attention. I have not added such euro filters, but it could help. You can refer to https://github.com/mkocabas/VIBE/blob/851f779407445b75cd1926402f61c931568c6947/demo.py#L229

For our implementation, we just trained the VAE part much longer for better motion quality. You can set it to 3000 or 4000 epochs.

cyk990422 commented 1 year ago

Thank you for your timely reply. Have you tried to use only rot6D as input for training? Is there any motion jitter?

ChenFengYe commented 1 year ago

I have tried rot6d in the early research, which is much smooth compared to the rotation vector. Rot6d even is the greatest representation for the motion-only generation, I believe, to avoid motion jitter. However, it could cause global artifacts, like foot-sliding on the ground. If you do not care about that, rot6d is great.

cyk990422 commented 1 year ago

Thank you very much for your patience, I will try it