search

ZhengyiLuo / PULSE

Official Implementation of the ICLR 2024 spotlight paper: Universal Humanoid Motion Representations for Physics-Based Control
https://www.zhengyiluo.com/PULSE
149 stars 6 forks source link

Error When Running Training Script from SMPLX Tasks Section in README #15

Open zxf8665905 opened 1 week ago

zxf8665905 commented 1 week ago

After running the training script from the SMPLX Tasks section in the README, an error occurred with the following message.

command:

python phc/run_hydra.py env.task=HumanoidSpeedZ env=env_pulsex_amp exp_name=pulse_x_speed robot=smplx_humanoid learning=pulse_z_task env.models=['output/HumanoidIm/pulse_vae_x/Humanoid.pth'] env.motion_file=sample_data/amass_isaac_x_simple_run.pkl

error message:

Importing module 'gym_38' (/opt/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /opt/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 2.4.1+cu121
Device count 1
/opt/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/gymuser/.cache/torch_extensions/py38_cu121 as PyTorch extensions root...
Emitting ninja build file /home/gymuser/.cache/torch_extensions/py38_cu121/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
2024-11-17 14:50:24,424 - INFO - logger - logger initialized
MOVING MOTION DATA TO GPU, USING CACHE: False
MOVING MOTION DATA TO GPU, USING CACHE: False
Importing module 'rlgpu_38' (/opt/isaacgym/python/isaacgym/_bindings/linux-x86_64/rlgpu_38.so)
/home/gymuser/.local/lib/python3.8/site-packages/wandb/util.py:116: SentryHubDeprecationWarning: `sentry_sdk.Hub` is deprecated and will be removed in a future major release. Please consult our 1.x to 2.x migration guide for details on how to migrate `Hub` usage to the new API: https://docs.sentry.io/platforms/python/migration/1.x-to-2.x
  sentry_hub = sentry_sdk.Hub(sentry_client)
wandb: Currently logged in as: xuefeng0578 (xuefeng0578-github). Use `wandb login --relogin` to force relogin
wandb: wandb version 0.18.7 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.12.21
wandb: Run data is saved locally in /home/zxf/dev/robot/PULSE/wandb/run-20241117_145027-xl92fdwf
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run sunny-voice-10
wandb: ⭐️ View project at https://wandb.ai/xuefeng0578-github/pulse
wandb: 🚀 View run at https://wandb.ai/xuefeng0578-github/pulse/runs/xl92fdwf
wandb: WARNING Calling wandb.run.save without any arguments is deprecated.Changes to attributes are automatically persisted.
torch_deterministic: False
torch_deterministic: False
torch_deterministic: False
Setting seed: 0
Started to train
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid.py:781: DeprecationWarning: an integer is required (got type isaacgym._bindings.linux-x86_64.gym_38.DofDriveMode).  Implicit conversion to integers using __int__ is deprecated, and may be removed in a future version of Python.
  asset_options.default_dof_drive_mode = gymapi.DOF_MODE_NONE
Humanoid Weights [73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507, 73.56829679687507]

****************************** Current motion keys ******************************
Sampling motion: tensor([ 61,  93, 181,  16, 150, 146, 191, 220,  11,   0, 248,  59,  88,  50,
         45, 198,  45,  82, 202,  35, 158, 101, 210,  60, 115, 205, 121, 111,
        117, 131], device='cuda:0')
['KIT_205_walking_medium10_stageii' 'KIT_7_WalkingStraightForwards06_stageii' 'KIT_6_WalkInClockwiseCircle01_1_stageii' 'KIT_9_run01_stageii' 'KIT_314_walking_run03_stageii' 'ACCAD_Female1Walking_c3d_B5_-_walk_backwards_stageii' 'KIT_348_run01_stageii' 'KIT_424_seesaw_backwards10_stageii' 'KIT_183_walking_fast05_stageii' 'ACCAD_Male2Running_c3d_C8_-_run_backwards_to_stand_stageii' 'ACCAD_Female1Running_c3d_C25_-__side_step_right_stageii' 'ACCAD_Male2Running_c3d_C3_-_run_stageii' 'KIT_183_walking_fast04_stageii' 'KIT_348_walking_medium07_stageii' 'KIT_317_walking_medium09_stageii' 'KIT_317_walking_medium04_stageii' 'KIT_317_walking_medium09_stageii' 'KIT_7_WalkingStraightForwards09_stageii' 'KIT_348_walking_medium03_stageii' 'KIT_12_WalkInClockwiseCircle09_stageii' 'KIT_425_walking_fast02_stageii' 'KIT_9_WalkInClockwiseCircle10_stageii' 'KIT_6_WalkInCounterClockwiseCircle06_1_stageii' 'KIT_7_WalkInCounterClockwiseCircle09_stageii' 'KIT_317_run03_stageii' 'KIT_348_walking_medium10_stageii' 'ACCAD_Male1Running_c3d_Run_C24_-_quick_side_step_left_stageii' 'KIT_348_walking_fast04_stageii' 'KIT_675_walk_with_handrail_beam_right07_stageii' 'KIT_11_WalkingStraightForwards03_stageii'] .....
*********************************************************************************

/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
/home/zxf/dev/robot/PULSE/poselib/poselib/skeleton/skeleton3d.py:1103: DeprecationWarning: Please use `gaussian_filter1d` from the `scipy.ndimage` namespace, the `scipy.ndimage.filters` namespace is deprecated.
  velocity = torch.from_numpy(filters.gaussian_filter1d(velocity, 2, axis=-3, mode="nearest")).to(p)
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:04<00:00,  3.10it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1536/1536 [00:00<00:00, 574244.67it/s]
Loaded 1536 motions with a total length of 11141.902s and 335793 frames.
=> loading checkpoint 'output/HumanoidIm/pulse_vae_x/Humanoid.pth'
/home/gymuser/.local/lib/python3.8/site-packages/rl_games/algos_torch/torch_ext.py:56: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  return func(*args, **kwargs)
/home/gymuser/.local/lib/python3.8/site-packages/gym/spaces/box.py:84: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
RL device:  cuda:0
1536
48
781
0
Error executing job with overrides: ['env.task=HumanoidSpeedZ', 'env=env_pulsex_amp', 'exp_name=pulse_x_speed', 'robot=smplx_humanoid', 'learning=pulse_z_task', 'env.models=[output/HumanoidIm/pulse_vae_x/Humanoid.pth]', 'env.motion_file=sample_data/amass_isaac_x_simple_run.pkl']
Traceback (most recent call last):
  File "phc/run_hydra.py", line 347, in main
    runner.run(cfg)
  File "/home/gymuser/.local/lib/python3.8/site-packages/rl_games/torch_runner.py", line 139, in run
    self.run_train()
  File "/home/gymuser/.local/lib/python3.8/site-packages/rl_games/torch_runner.py", line 122, in run_train
    agent = self.algo_factory.create(self.algo_name, base_name='run', config=self.config)
  File "/home/gymuser/.local/lib/python3.8/site-packages/rl_games/common/object_factory.py", line 15, in create
    return builder(**kwargs)
  File "phc/run_hydra.py", line 258, in <lambda>
    runner.algo_factory.register_builder('amp', lambda **kwargs: amp_agent.AMPAgent(**kwargs))
  File "/home/zxf/dev/robot/PULSE/phc/learning/amp_agent.py", line 39, in __init__
    super().__init__(base_name, config)
  File "/home/zxf/dev/robot/PULSE/phc/learning/common_agent.py", line 37, in __init__
    a2c_common.A2CBase.__init__(self, base_name, config)
  File "/home/gymuser/.local/lib/python3.8/site-packages/rl_games/common/a2c_common.py", line 85, in __init__
    self.vec_env = vecenv.create_vec_env(self.env_name, self.num_actors, **self.env_config)
  File "/home/gymuser/.local/lib/python3.8/site-packages/rl_games/common/vecenv.py", line 282, in create_vec_env
    return vecenv_config[vec_env_name](config_name, num_actors, **kwargs)
  File "phc/run_hydra.py", line 250, in <lambda>
    vecenv.register('RLGPU', lambda config_name, num_actors, **kwargs: RLGPUEnv(config_name, num_actors, **kwargs))
  File "phc/run_hydra.py", line 201, in __init__
    self.full_state["obs"] = self.reset()
  File "phc/run_hydra.py", line 218, in reset
    self.full_state["obs"] = self.env.reset(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/vec_task_wrappers.py", line 55, in reset
    self.task.reset(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid.py", line 531, in reset
    self._reset_envs(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp_task.py", line 74, in _reset_envs
    super()._reset_envs(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp.py", line 352, in _reset_envs
    super()._reset_envs(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid.py", line 577, in _reset_envs
    self._reset_actors(env_ids) # this funciton calle _set_env_state, and should set all state vectors
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp.py", line 361, in _reset_actors
    self._reset_ref_state_init(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_speed.py", line 248, in _reset_ref_state_init
    super()._reset_ref_state_init(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp.py", line 469, in _reset_ref_state_init
    motion_ids, motion_times, root_pos, root_rot, dof_pos, root_vel, root_ang_vel, dof_vel, rb_pos, rb_rot, body_vel, body_ang_vel = self._sample_ref_state(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_speed.py", line 252, in _sample_ref_state
    motion_ids, motion_times, root_pos, root_rot, dof_pos, root_vel, root_ang_vel, dof_vel, rb_pos, rb_rot, body_vel, body_ang_vel  = super()._sample_ref_state(env_ids)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp.py", line 460, in _sample_ref_state
    root_pos, root_rot, dof_pos, root_vel, root_ang_vel, dof_vel, rb_pos, rb_rot, body_vel, body_ang_vel = self._get_fixed_smpl_state_from_motionlib(motion_ids, motion_times, curr_gender_betas)
  File "/home/zxf/dev/robot/PULSE/phc/env/tasks/humanoid_amp.py", line 422, in _get_fixed_smpl_state_from_motionlib
    vertices_curr, joints_curr = self.smpl_parser_n.get_joints_verts(poses_curr, betas_curr, root_pos_curr)
  File "/home/gymuser/.local/lib/python3.8/site-packages/smpl_sim/smpllib/smpl_parser.py", line 424, in get_joints_verts
    smpl_output = self.forward(
  File "/home/gymuser/.local/lib/python3.8/site-packages/smpl_sim/smpllib/smpl_parser.py", line 407, in forward
    smpl_output = super(SMPLX_Parser, self).forward(*args, **kwargs)
  File "/home/gymuser/.local/lib/python3.8/site-packages/smplx/body_models.py", line 1335, in forward
    vertices, joints = lbs(
  File "/home/gymuser/.local/lib/python3.8/site-packages/smplx/lbs.py", line 205, in lbs
    v_shaped = v_template + blend_shapes(betas, shapedirs)
  File "/home/gymuser/.local/lib/python3.8/site-packages/smplx/lbs.py", line 291, in blend_shapes
    blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])
  File "/home/gymuser/.local/lib/python3.8/site-packages/torch/functional.py", line 381, in einsum
    return einsum(equation, *_operands)
  File "/home/gymuser/.local/lib/python3.8/site-packages/torch/functional.py", line 386, in einsum
    return _VF.einsum(equation, operands)  # type: ignore[attr-defined]
RuntimeError: einsum(): subscript l has size 20 for operand 1 which does not broadcast with previously seen size 30

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.