search

GuyTevet / motion-diffusion-model

The official PyTorch implementation of the paper "Human Motion Diffusion Model"
MIT License
2.96k stars 315 forks source link

VPoser Inverse Kinematics Engine #23

Open nghorbani opened 1 year ago

nghorbani commented 1 year ago

Hi, Thank you for sharing this amazing work! I see that you are using an old version of GMM prior for joint2smpl IK. There is a more advanced alternative introduced in SMPLify-X; i.e. VPoser. I have already adapted the code, please have a look. The VPoser IK works on batches on GPU and the output is exactly as AMASS and can be directly played in Blender via Blender SMPL-X addon. I believe this setup includes less hassle compared too creating FBX directly from pkl files that people are already doing. Moreover, using the blender addon one cn also create FBX files. Keep up the great work. Best, Nima

GuyTevet commented 1 year ago

Thanks @nghorbani for this code contribution! I will look deeper into that and use it in our code. Just a question - HumanML3D uses SMPL positions, shouldn't we prefer optimization to SMPL joints (rather than SMPL-X)?

nghorbani commented 1 year ago

There is actually no body model restriction; i.e. the human_body_prior smpl loader will automatically use correct body pose parameters and vposer should already yield good results.

GuyTevet commented 1 year ago

Cool thanks! I will have a chance to integrate sometime during November. In case you want to send a pull request sooner that will be wonderful! (and you will be listed as a contributor so that's a win-win:) )

cjerry1243 commented 1 year ago

@GuyTevet I am curious why you are converting the motion representation to joint positions for IK retargeting. I feel that it might be more straightforward to use the rotation6d data and drive the SMPL character. What could be the differences between the results using the two methods?

Also, I don't get the idea why a redundant motion representation has to be used. I believe the same representation is used for calculating the evaluation metrics. I feel that the redundancy may affect the evaluation results.

mayank64ce commented 7 months ago

Hey @nghorbani I don't know what I am doing wrong but I am getting this rendering:

https://drive.google.com/file/d/1qWnBMS52CSd5yRf_jEWElo8FzZXWiSjC/view?usp=drive_link

for this sequence:

https://github.com/GuyTevet/motion-diffusion-model/assets/43180708/1e38542c-b6e8-4cf6-baa9-ae781af7e660