Example script not reponsive

[RuiningLi]

Hi Kevin,

Super nice to see this library.

I was trying to play around the IK solver for Unitree G1 humanoid. It seems like the mocap data which should be set as the target pos / quat was overwritten somewhere to be the same as the original humanoid pose, as a result, the initial G1 pose doesn't get altered by doing IK? See the image below.

Since I'm quite new to Mujoco, it can also be the case that I'm doing something not intended. Could you clarify a bit more on what the code is supposed to do?

Many thanks!

[kevinzakka]

Hi @RuiningLi, thanks for using mink!

The example scripts are meant to be interactive! The mocap bodies are initialized at the current configuration. The point is that you can double click the mocap body of interest, then use ctrl + left click to rotate and ctrl + right click to translate. This will change the targets for IK and you should see the configuration change in response.

[RuiningLi]

Interesting! I wonder if there is a way to directly set the target of those locations? I suppose I can just comment out these lines?

        for hand, foot in zip(hands, feet):
            mink.move_mocap_to_frame(model, data, f"{foot}_target", foot, "site")
            mink.move_mocap_to_frame(model, data, f"{hand}_target", hand, "site")
        data.mocap_pos[com_mid] = data.subtree_com[1]

It also looks to me that the end configuration does not necessarily satisfy the contact constraint, e.g., the hand of the G1 penetrate the body. Is this just because the targeted pose is too extreme? (Would a different optimization method, e.g., gradient descent help with this issue?)

[kevinzakka]

Hi @RuiningLi, so this specific example does not simulate the entire physics pipeline. It just performs IK and shows you the result of mj_forward. You would have to integrate the velocities into joint positions and feed that to position actuators and call mj_step to fully exercise the physics pipeline (which includes collision detection for example). See the Spot quadruped example for how this is done.

That being said, mink does support collision avoidance as a constraint, it's just not enabled in the G1 example. Check out the UR5e example or the new ALOHA example to see how it can be enabled and used during the IK solve.

[RuiningLi]

Ahh yeah it makes a lot of sense! Thank you for your responses and the amazing work!