Error when load mjcf file for mujoco

[JYChen18]

I wanted to load the mjcf model from https://github.com/deepmind/mujoco_menagerie but failed.

I tried pip install pytorch-kinematics and when I ran pk.build_chain_from_mjcf, I met the following error

File "test.py", line 42, in __init__          
    self.chain = pk.build_chain_from_mjcf(open(mjcf_path).read()).to(dtype=torch
.float, device=device)
  File "miniconda3/lib/python3.8/site-packages/pytorch_kinematics/m
jcf.py", line 68, in build_chain_from_mjcf
    _build_chain_recurse(root_frame, root_body)
  File "miniconda3/lib/python3.8/site-packages/pytorch_kinematics/m
jcf.py", line 26, in _build_chain_recurse
    parent_frame.link.visuals = geoms_to_visuals(parent_body.geom)
  File "miniconda3/lib/python3.8/site-packages/pytorch_kinematics/m
jcf.py", line 20, in geoms_to_visuals
    raise ValueError('Invalid geometry type %s.' % g.type)
ValueError: Invalid geometry type None.

Then I tried pip install "pytorch-kinematics[mujoco]" as in README.md, but I got WARNING: pytorch-kinematics 0.5.6 does not provide the extra 'mujoco' during installation. I ignored the warning and ran pk.build_chain_from_mjcf again, but I got the same error as before.

Even if I removed the line which raised the previous error, I met another error

  File "miniconda3/lib/python3.8/site-packages/pytorch_kinematics/mjcf.py", line 70, in build_chain_from_mjcf
    _build_chain_recurse(root_frame, root_body)
  File "miniconda3/lib/python3.8/site-packages/pytorch_kinematics/mjcf.py", line 36, in _build_chain_recurse
    joint_type=JOINT_TYPE_MAP[joint.type])
KeyError: None

How can I load the mjcf model for mujoco correctly? I would appreciate any help. Thanks so much in advance.

[PeterMitrano]

Thanks for asking, my guess is as the error says there's some mujoco geom type that we didn't implement. Can you say which model from menagerie you're trying to load?

[JYChen18]

Thanks for your reply. I want to use the shadow hand from https://github.com/deepmind/mujoco_menagerie/blob/main/shadow_hand/right_hand.xml

[PeterMitrano]

Thanks for repoting this issue -- now that the ICRA deadline has passed I've had a chance to look at this. My fix is just to pass-through all types of geoms, since pytorch kinematics does not actually use this information for any calculations. As for the joint type, we can fix that by dropping dm_control.mjcf in favor of just directly using mujoco. The problem was that the menagerie models make heavy use of the "class" XML mechanism, which mjcf wasn't happy about. Much easier to just use mujoco.MjModel directly which has all the info we need.

I believe I've fixed the issue and we'll push a new version hopefully this week. I tested with this:

    with open('right_hand.xml') as f:
        xml = f.read()
    chain = pk.build_chain_from_mjcf(xml)
    print(chain.get_frame_names())
    print(chain.get_joint_parameter_names())
    th = np.zeros(len(chain.get_joint_parameter_names()))
    fk_dict = chain.forward_kinematics(th, end_only=True)
    print(fk_dict['rh_lfproximal'])
    print(fk_dict['rh_lfdistal'])

and the numbers look right based on visual inspection!

[JYChen18]

Thanks for your effort! I finally used another mjcf model in my experiments, but I may try your update later.