Use phc_kp_mcp_iccv for Language-to-motion Demo

[yw0208]

Hi, another issue. When I run "python phc/run_hydra.py learning=im_mcp exp_name=phc_kp_mcp_iccv env=env_im_getup_mcp env.task=HumanoidImMCPDemo robot=smpl_humanoid robot.freeze_hand=True robot.box_body=False env.z_activation=silu env.motion_file=sample_data/amass_isaac_standing_upright_slim.pkl env.models=['output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth'] env.num_envs=1 env.obs_v=7 headless=False epoch=-1 test=True no_virtual_display=True",

I noticed that you use the single primitive model. From what I've observed, it's worse than full model.

So I changed env.models and used 'output/HumanoidIm/phc_kp_mcp_iccv/Humanoid.pth', and following is the error:

....... File "PHC_hydra/phc/env/tasks/humanoid_im_mcp.py", line 27, in init self.pnn = load_pnn(pnn_ck, num_prim = self.num_prim, has_lateral = self.has_lateral, activation = self.z_activation, device = self.device) File "/PHC_hydra/phc/learning/network_loader.py", line 63, in load_pnn mlp_args = {'input_size': state_dict_load['a2c_network.pnn.actors.0.0.weight'].shape[1], 'units':layer_size[:-1], 'activation': activation, 'dense_func': torch.nn.Linear} KeyError: 'a2c_network.pnn.actors.0.0.weight'

[ZhengyiLuo]

"python phc/run_hydra.py learning=im_mcp exp_name=phc_kp_mcp_iccv env=env_im_getup_mcp env.task=HumanoidImMCPDemo robot=smpl_humanoid robot.freeze_hand=True robot.box_body=False env.z_activation=silu env.motion_file=sample_data/amass_isaac_standing_upright_slim.pkl env.models=['output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth'] env.num_envs=1 env.obs_v=7 headless=False epoch=-1 test=True no_virtual_display=True"

is using the multiple primitive model. The phc_kp_mcp_iccv flag is pointing to the MCP composer, while output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth is pointing to the trained PNN network. If the humanoid can get up from the ground, it has multiple primitives.

[yw0208]

"python phc/run_hydra.py learning=im_mcp exp_name=phc_kp_mcp_iccv env=env_im_getup_mcp env.task=HumanoidImMCPDemo robot=smpl_humanoid robot.freeze_hand=True robot.box_body=False env.z_activation=silu env.motion_file=sample_data/amass_isaac_standing_upright_slim.pkl env.models=['output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth'] env.num_envs=1 env.obs_v=7 headless=False epoch=-1 test=True no_virtual_display=True"

is using the multiple primitive model. The phc_kp_mcp_iccv flag is pointing to the MCP composer, while output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth is pointing to the trained PNN network. If the humanoid can get up from the ground, it has multiple primitives.

But I indeed feel the performance gets worse. Here is an example. The first video shows the results output by the latest PHC and the second shows the output by the past version of PHC.

python phc/run_hydra.py learning=im_mcp exp_name=phc_kp_mcp_iccv env=env_im_getup_mcp env.task=HumanoidImMCPDemo robot=smpl_humanoid robot.freeze_hand=True robot.box_body=False env.z_activation=silu env.motion_file=sample_data/amass_isaac_standing_upright_slim.pkl env.models=['output/HumanoidIm/phc_kp_pnn_iccv/Humanoid.pth'] env.num_envs=1 env.obs_v=7 headless=False epoch=-1 test=True no_virtual_display=True

https://github.com/ZhengyiLuo/PerpetualHumanoidControl/assets/70508717/a397e0cd-0dc5-43bd-b283-5d0675f3bf40

python phc/run.py --task HumanoidImMCPDemo --cfg_env phc/data/cfg/phc_kp_mcp_iccv.yaml --cfg_train phc/data/cfg/train/rlg/im_mcp.yaml --motion_file sample_data/amass_isaac_standing_upright_slim.pkl --network_path output/phc_kp_mcp_iccv --test --num_envs 1 --epoch -1  --no_virtual_display

https://github.com/ZhengyiLuo/PerpetualHumanoidControl/assets/70508717/b0e0ef03-7581-4cab-bf06-040678fd8ffb

[ZhengyiLuo]

Oh that's a bug! Thanks for pointing it out!

I have fixed it in the newest push. I used the wrong activation function (relu instead of silu) for the ICCV models. It has now being fixed. Going to release PHC+ soon, which will use the new activation function.

[ZhengyiLuo]

env.z_activation=relu is the change in the command.