search

lisiyao21 / Bailando

Code for CVPR 2022 paper "Bailando: 3D dance generation via Actor-Critic GPT with Choreographic Memory"
Other
382 stars 59 forks source link

How to show the dancing character via Mixamo? #16

Open LinghaoChan opened 2 years ago

LinghaoChan commented 2 years ago

According to the code you provided, I can only get the dancing stickman (skeleton), how to get the dancing character animation. Is there any relevant code or tutorial about Mixamo?

Thanks a lot.

lisiyao21 commented 2 years ago

Hi, thank you for your interest!

To drive character animation, you need to transfer 3D joint keypoints to rotation matrix/eular angles (this step is called "inverse kinemetics", IK), save them into some format like .fbx and render the character using CG softwares like Blender. The IK step was implemented in SenseTime (a company) with Unity software, which is not for open source.

We realize that the public is willing to have a one for avatar in a simpler way. We will try to train and release a "rotmat" version in few months.

aleeyang commented 2 years ago

According to the code you provided, I can only get the dancing stickman (skeleton), how to get the dancing character animation. Is there any relevant code or tutorial about Mixamo?

Thanks a lot.

I have find a project (video2bhv)which can convert 3d joint kepoint into a motion file(bvh) which can control the mixamo in blender.But I get confused motion sequences.Anyway, it is still possible way to visualize your result in blender. If you are interested in it ,you can have a try.. by the way, if there are any change in code(video2bvh) which can make the result better.Wish you can share with me. Thank you a lot.

iooops commented 9 months ago

Hi, thank you for your interest!

To drive character animation, you need to transfer 3D joint keypoints to rotation matrix/eular angles (this step is called "inverse kinemetics", IK), save them into some format like .fbx and render the character using CG softwares like Blender. The IK step was implemented in SenseTime (a company) with Unity software, which is not for open source.

We realize that the public is willing to have a one for avatar in a simpler way. We will try to train and release a "rotmat" version in few months.

So is the "rotmat" version available now?