How to deploy

[kassajing]

I trained the model using the 1-20 data. How should I deploy it to a real robotic arm?

[j96w]

Hi @kassajing, glad that you have successfully gone through the data processing and training parts!! The learned policy outputs 46-dim action space consisting of (in order):

• 3-dim right arm end-effector translation
• 3-dim left arm end-effector translation
• 4-dim right arm end-effector orientation in quaternion
• 4-dim left arm end-effector orientation in quaternion
• 16-dim right LEAP hand joint positions
• 16-dim left LEAP hand joint positions

You will need a real robot to verify the learned policy. The LEAP hand is open-sourced and can be built by following the LEAP manual. You can use the robot arm you have, and use end-effector position control with the 7-dim translation and orientation outputs.

[kassajing]

Thank you very much for your reply！I have a new question need your help. Regarding the """....layers.0.weight" and """...layers.0.bias" in the model, the dimension of bias is (32,) (64,) (128,) (256, ), does this part correspond to the joints of the 16-dim LEAP hand? Does (32,6,1) (64,,32,1) (128,64,1) (256,128,1) in weights correspond to the arm? Which dimension of data corresponds to 3-dim or 4-dim?

[j96w]

I think you are referring to the weights of the MLP layers, which are not the final action outputs. Checkout this function to see how the action is generated. The action format (46-dim action output) aligns with the action output in this function.