LeCAR-Lab / human2humanoid

[IROS 2024] Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation. [CoRL 2024] OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
https://omni.human2humanoid.com/
276 stars 13 forks source link

关于状态空间的版本和实机部署 #14

Closed Leeeeeeeeeeeeeeeeeeeeeeeeeeeeeo closed 3 weeks ago

Leeeeeeeeeeeeeeeeeeeeeeeeeeeeeo commented 1 month ago

大佬你好,请问这么多版本的obs各自的作用是什么? image

我已经使用了经过过滤后的数据集训练的privileged teacher policy,接下来如果要实机部署的话是使用以下参数吗 --config-name=config_teleop task=h1:teleop run_name=OmniH2O_STUDENT env.num_observations=1665 env.num_privileged_obs=1742 motion.teleop_obs_version=v-teleop-extend-vr-max-nolinvel motion.teleop_selected_keypoints_names=[] motion.extend_head=True num_envs=4096 asset.zero_out_far=False asset.termination_scales.max_ref_motion_distance=1.5 sim_device=cuda:0 motion.motion_file=resources/motions/h1/stable_punch.pkl rewards=rewards_teleop_omnih2o_teacher rewards.penalty_curriculum=True rewards.penalty_scale=0.5 train.distill=True train.policy.init_noise_std=0.001 env.add_short_history=True env.short_history_length=25 noise.add_noise=False noise.noise_level=0 train.dagger.load_run_dagger=TEACHER_RUN_NAME train.dagger.checkpoint_dagger=XXX train.dagger.dagger_only=True

我看到最下面的 H2O Policy (8point tracking, no history, MLP, with linear velocity in the state space)是没有线速度的,而且没有对髋关节进行跟踪,这个Policy能否部署到实机上呢?

TairanHe commented 3 weeks ago

Hi你好,obs版本多的原因是因为我开发过程中迭代了非常多的状态空间选择。H2O policy是可以部署到实机的。感谢!

Leeeeeeeeeeeeeeeeeeeeeeeeeeeeeo commented 5 days ago

感谢大佬回复! 另外我想请教一下: (1)项目中提到的使用HybrIK进行人体位姿估计,得到的SMPL格式的24*3维关键点位置是可以作为H2O Policy的输入的吗? (2)如何修改HybrIK使其可以实时估计人体位姿,并且达到30Hz,我目前使用4080s显卡,处理速度是3.42it/s

TairanHe commented 4 days ago
  1. H2O是8点(肩、肘、手,脚踝)输入,你可以选择SMPL的关键点(肩、肘、手,脚踝)喂给H2O
  2. 我当时实验是4090,可以跑到30hz
Leeeeeeeeeeeeeeeeeeeeeeeeeeeeeo commented 4 days ago

好的,谢谢大佬!