search

GuyTevet / motion-diffusion-model

The official PyTorch implementation of the paper "Human Motion Diffusion Model"
MIT License
3.16k stars 346 forks source link

VPoser Inverse Kinematics Engine #23

Open nghorbani opened 2 years ago

nghorbani commented 2 years ago

Hi, Thank you for sharing this amazing work! I see that you are using an old version of GMM prior for joint2smpl IK. There is a more advanced alternative introduced in SMPLify-X; i.e. VPoser. I have already adapted the code, please have a look. The VPoser IK works on batches on GPU and the output is exactly as AMASS and can be directly played in Blender via Blender SMPL-X addon. I believe this setup includes less hassle compared too creating FBX directly from pkl files that people are already doing. Moreover, using the blender addon one cn also create FBX files. Keep up the great work. Best, Nima

GuyTevet commented 2 years ago

Thanks @nghorbani for this code contribution! I will look deeper into that and use it in our code. Just a question - HumanML3D uses SMPL positions, shouldn't we prefer optimization to SMPL joints (rather than SMPL-X)?

nghorbani commented 2 years ago

There is actually no body model restriction; i.e. the human_body_prior smpl loader will automatically use correct body pose parameters and vposer should already yield good results.

GuyTevet commented 2 years ago

Cool thanks! I will have a chance to integrate sometime during November. In case you want to send a pull request sooner that will be wonderful! (and you will be listed as a contributor so that's a win-win:) )

cjerry1243 commented 2 years ago

@GuyTevet I am curious why you are converting the motion representation to joint positions for IK retargeting. I feel that it might be more straightforward to use the rotation6d data and drive the SMPL character. What could be the differences between the results using the two methods?

Also, I don't get the idea why a redundant motion representation has to be used. I believe the same representation is used for calculating the evaluation metrics. I feel that the redundancy may affect the evaluation results.

mayank64ce commented 1 year ago

Hey @nghorbani I don't know what I am doing wrong but I am getting this rendering:

https://drive.google.com/file/d/1qWnBMS52CSd5yRf_jEWElo8FzZXWiSjC/view?usp=drive_link

for this sequence:

https://github.com/GuyTevet/motion-diffusion-model/assets/43180708/1e38542c-b6e8-4cf6-baa9-ae781af7e660