search

ChenFengYe / motion-latent-diffusion

[CVPR 2023] Executing your Commands via Motion Diffusion in Latent Space, a fast and high-quality motion diffusion model
https://chenxin.tech/mld/
MIT License
586 stars 55 forks source link

About performance #33

Open layumi opened 1 year ago

layumi commented 1 year ago

Thanks a lot for the great work. I test the trained model "1222_mld_humanml3d_FID041.ckpt"

The performance is as follows, which seems a bit lower than the reported performance in the paper.

image

Is the performance correct?

ChenFengYe commented 1 year ago

Hi, thanks for your attention! Your testing on R_Precision is a bit lower, 0.756 (your testing) v.s. 0.772 (Tab.1 in Paper). However, it seems your FID is even better, 0.468(your testing) v.s. 0.473(Tab.1 in Paper). Yet, I believe your testing is quite close.

BTW, the FID of this model "1222_mld_humanml3d_FID041.ckpt" should be 0.41, like the name. The mean/std files could cause this downgrade. Please pay attention to the following: https://github.com/ChenFengYe/motion-latent-diffusion/blob/c28a06435077800fde4d76ef93eb2a4016a5120c/README.md?plain=1#L196

layumi commented 1 year ago

Thanks a lot @ChenFengYe ^_^

I am checking the dataset.

Could I have your paper model with 47.3 to evaluate as well?

Cheers