Franklin2002-Peng
commented
2 months ago Hi Nick, I also have a puzzle about this part. According to my understanding, given a in-the-wild video, firstly the pose of human hand and object is reconstructed using the modified MCC-HO so that {(hand_pose_k,object_pose_k)} sequence is obtained. Then retargeting is performed by optimizing equation (1) step by step. However, I have two puzzles about this: 1. using equation (1) to get a[k] and run simulation for one-step to get o[k]. I guess the pose in o[k] might have deviation with original {(hand_pose_k,object_pose_k)} sequence. But a[k+1] is generated from (hand_pose_k+1,object_pose_k+1) rather than o[k]. In that case, the final {(a[k],o[k])} might not be very consistent? 2.it seems that the objective in equation (1) only contain hand pose term, without the object pose part. would this leads to only learn the hand prior? Best, Yuanfang
Hey!
Congratulations on this awesome work!
I have a question related to https://github.com/hgaurav2k/hop/issues/1. I am curious about the mapping of the hand pose to the robot end effector, specifically Eq.1 in the paper. If I understand the code correctly, the pre-training step refers to only the causal transformer model training and not the optimization in Eq. 1. Thus, I am wondering, do you plan to release this part of your work as well? I would be really curious :)
Many Thanks and Cheers, -Nick