search

PKU-EPIC / UniDexGrasp

Official code for "UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy" (CVPR 2023)
102 stars 10 forks source link

Possible Bug in pointcloud observation. #18

Closed yangyichu closed 6 months ago

yangyichu commented 7 months ago

PointCloud in observation is:

https://github.com/PKU-EPIC/UniDexGrasp/blob/c3f0a39c67572acf7dc87c381cd2ed37c434430d/dexgrasp_policy/dexgrasp/tasks/shadow_hand_random_load_vision.py#L1048-L1053

Above code gives obs as [.....| point_clouds_xyz(1024*6)| mask_hand(1024)| mask_object(1024)|...], for example, [....|(x1,y1,z1, r1,g1,b1),(x2,y2,z2,r2,g2,b2)....|(hm1,hm2,....)|(om1,om2....)|.....] But when reshaping point cloud:

Yet in student policy: https://github.com/PKU-EPIC/UniDexGrasp/blob/c3f0a39c67572acf7dc87c381cd2ed37c434430d/dexgrasp_policy/dexgrasp/algorithms/rl/ppo/module.py#L165

This is only right if pointcloud is dim-8, which requires the obs to be [.....| point_cloud_xyz_hm_om(1024*8)|...] in which point_cloud_xyz_hm_om is torch.cat([pointcloud_xyz, hand_mask.float().unsqueeze(-1), object_mask.float().unsqueeze(-1)], dim=-1), for example [....|(x1,y1,z1,r1,g1,b1,hm1,om1),(x2,y2,z2,r2,b2,g2,hm2,om2)|....]

wkwan7 commented 6 months ago

Thank you for pointing out this issue! This was an error in our code organization during a previous revision, and it has been updated in the latest version.