search

google-deepmind / mujoco

Multi-Joint dynamics with Contact. A general purpose physics simulator.
https://mujoco.org
Apache License 2.0
7.77k stars 771 forks source link

How to incoporate Motion Capture(MoCap) data ground truth into the system? #1814

Closed coder-rakeshpaul closed 3 days ago

coder-rakeshpaul commented 1 month ago

Hi,

I want to add the MoCap data ground truth to my environment. Similar to what has been incorporated in the dm_control environment. Where they have added to ground truth of the CMU MoCap data.

I find that they download the data from the google server and not sure how they actually mapped the model.

When we import the cmu_humanoid xml into the environment. We see that the model is floating in the air, so mapping any the joint angles the model does the motion (but it's feet is always above the ground) image

I want the feet to be on the ground so as to get the proper ground truth before training a RL policy.

How do I go about doing this?

PS: I don't just want to use the CMU Mocap data and want to add other mocap data hence not using the already existing ground truth of the CMU Mocap.

yuvaltassa commented 1 month ago

data.qpos[2] -= height_offset ?

coder-rakeshpaul commented 1 month ago

yes that was my 1st thought too, but what if at the start of the motion the person is not on his feet rather on his toes? This will cause the foot to go below the ground (since I was talking about a dataset there would be multiple clips and a common offset would have been nice), yes this seems like a specific case and could be manually tuned.

Do you think the dataset in dm_control's MoCap was done the same way (with the height offset you mentioned)? or could they have used a better more efficient method (which I maybe not aware off)?

Also another question, how do I find out how far above the ground the feet are? (to get the height offset) is there a function or something which gives the location of the feet wrt ground?

Balint-H commented 1 month ago

You can do a preprocessing step checking the height of the toes and heel throughout the animation. Then, subtract the minimum of that from the root height (+- some fixed offset to account for the foot thickness).

To get global position of a mujoco body in the world cartesian frame, you can check data.xpos[3*body_id], or data.body(body_name).xpos.