'scene_lib' argument missing in play_motion.py and error during training

[Minyoung1005]

Hi, thanks for releasing the code for Maskedmimic. Your work is awesome!

1

I am trying to reproduce the results and was replaying the .npy motion files provided in the repository, and just found an error as follows when I execute python phys_anim/scripts/play_motion.py phys_anim/data/motions/smpl_humanoid_walk.npy isaacgym smpl

omegaconf.errors.InterpolationKeyError: Interpolation key 'scene_lib' not found

After adding '+scene_lib=null' to the command string in play_motion.py, it works but I'm not sure if setting scene_lib as null is appropriate.

2

Also, when I run python phys_anim/train_agent.py +exp=full_body_tracker +backbone=isaacgym +robot=smpl motion_file=data/smpl/SMPL_NEUTRAL.pkl, I get the following error:

...
Loading object assets... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸  99% 0:00:01Found existing convex decomposition for mesh 'phys_anim/data/assets/scenes/train/SAMP/StraightChairs/68dc37f167347d73ea46bea76c64cc3d.obj'
Found existing convex decomposition for mesh 'phys_anim/data/assets/scenes/train/SAMP/HighStools/84c9082a8cbd03e5e76bc197b3a3ffc0.obj'
Found existing convex decomposition for mesh 'phys_anim/data/assets/scenes/train/SAMP/HighStools/613c706879ee39c9132c018af3949fe.obj'
Loading object assets... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
=========== Total number of unique objects is 339
Creating 4096 environments... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
=========== Building object playground
Spawning 613c706879ee39c9132c018af3949fe.urdf ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
/home/minyoung/Projects/ProtoMotions/phys_anim/utils/motion_lib.py:166: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  torch.tensor(key_body_ids, dtype=torch.long, device=device),
Loading motions from state file: data/smpl/SMPL_NEUTRAL.pkl
Error executing job with overrides: ['+exp=full_body_tracker', '+backbone=isaacgym', '+robot=smpl', 'motion_file=data/smpl/SMPL_NEUTRAL.pkl']
Error in call to target 'phys_anim.envs.mimic.isaacgym.MimicHumanoid':
InstantiationException('Error in call to target \'phys_anim.utils.motion_lib.MotionLib\':\nUnicodeDecodeError(\'utf-8\', b\'\\x00\......\x00\', 1224, 1225, \'invalid start byte\')\nfull_key: config.motion_lib')
full_key: env

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

How should I fix this error? I'm assuming that motion_file should be either .npy or .yaml, not .pkl, but I have no idea what file I should use for the first training stage (Train full body tracker: Run PYTHON_PATH phys_anim/train_agent.py +exp=full_body_tracker +backbone=isaacgym +robot=smpl motion_file=)

3

train_agent.py works when I pass an .npy file already existing in phys_anim/data. However, I get CUDA out of memory even though I've reduced the batch size a lot (16384 -> 128) to test the code in my local machine with a 3080Ti GPU with 16GB VRAM. Is there a way I can test the code on a local machine? I've tried adding algo.config.batch_size=128 argument when I was reducing the batch size. Should I change the number of the environments? If so, how can I do that?

4

When I go into the AMASS website, there are so many subdatasets available in various versions. What should I download?

Would appreciate your feedback, thanks!

[tesslerc]

1. Good catch. We were running play_motion.py with the --extra-args=... flag and manually providing some structs. Pushed a fix to the config structure so play_motion will auto-load the default settings. The user can override them, for example using --extra-args="+scenes=samp".
2. See the instructions in the README under data. It can receive .npy files, .yaml that points to multiple .npy files, or a packaged (pickled) MotionLib file in .pt format.
3. Try reducing the number of envs. Simpy add the flag num_envs=256 algo.config.batch_size=1024. You want to try and find the optimal working point, most envs and batch size but without it crashing due to out-of-memory errors.
4. For the SMPL one, download SMPL+H. For the SMPL-X body (with fingers), I've been working with SMPL-X G, although I believe the N one should also work.