Issue with convert_state.py

[kliu14146]

Hi all,

I'm trying to convert demonstrations for PickCube-v0 rgbd, with pd_ee_delta_pose controller. I followed the commands in scripts/example_demo_conversion/pick_place_single_object_envs.sh. The first command used to convert the controller space runs fine. The second command: python3 tools/convert_state.py --env-name PickCube-v0 --traj-name ../ManiSkill2-Learn/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose.h5 --json-name ../ManiSkill2-Learn/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose.json --output-name ../ManiSkill2-Learn/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose_rgbd.h5 --control-mode pd_ee_delta_pose --max-num-traj -1 --obs-mode rgbd --reward-mode dense returns the following error:

Traceback (most recent call last):
  File "tools/convert_state.py", line 244, in <module>
    main()
  File "tools/convert_state.py", line 227, in main
    convert_state_representation(keys, args, 0, os.getpid(), *extra_args)
  File "tools/convert_state.py", line 81, in convert_state_representation
    trajectory = GDict.from_hdf5(input_h5[key])
  File "/egrl/ManiSkill2-Learn/maniskill2_learn/utils/data/dict_array.py", line 325, in from_hdf5
    ret = load_hdf5(file, keys)
  File "/egrl/ManiSkill2-Learn/maniskill2_learn/utils/file/hdf5_utils.py", line 86, in load_hdf5
    ret = _load_hdf5(file, keys, only_one)
  File "/egrl/ManiSkill2-Learn/maniskill2_learn/utils/file/hdf5_utils.py", line 60, in _load_hdf5
    ret[key_value] = _load_hdf5(file[key], load_keys_i, only_one)
  File "/egrl/ManiSkill2-Learn/maniskill2_learn/utils/file/hdf5_utils.py", line 25, in _load_hdf5
    if keys[0].startswith("list"):
IndexError: list index out of range

If I print out the file and the file keys in _load_hdf5, it looks like the following files are being opened and the obs key object is an empty list (hence the error):

<HDF5 group "/traj_0" (4 members)> ['obs', 'actions', 'success', 'env_states']
<HDF5 group "/traj_0/obs" (0 members)> []

The first command does seem to run to completion, and it would make sense that the obs key has no members given that the saved state is "none"? I am saving the demos inside ManiSkill2-Learn/demos, and I am running this on a g4dn.xlarge ec2 instance. The h5py module is version 3.9.0, and I am using python 3.9. Please let me know if I am missing something simple - thank you!

[xuanlinli17]

This is caused by the fact that demo file is not found, since there is a typo in your command (traj, json, output should be under ManiSkill2, not in ManiSkill2-Learn).

The command should be python3 tools/convert_state.py --env-name PickCube-v0 --traj-name ../ManiSkill2/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose.h5 --json-name ../ManiSkill2/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose.json --output-name ../ManiSkill2/demos/v0/rigid_body/PickCube-v0/trajectory.none.pd_ee_delta_pose_rgbd.h5 --control-mode pd_ee_delta_pose --max-num-traj -1 --obs-mode rgbd --reward-mode dense

[kliu14146]

I moved the demos folder into ManiSkill2 (same directory level as ManiSkill2-Learn). I then ran your updated command, but I get the same error. It looks like the file is loaded, but the "obs" key is an empty list (hence the index out of range error). Does the entire chain of commands (convert control, then convert observation) need to be run in ManiSkill2?

[xuanlinli17]

I've fixed it in the latest ManiSkill2-Learn commit.