search

hyy0613 / RT-1

This is the completion of google's rt-1 project code and can run directly.
Apache License 2.0
22 stars 4 forks source link

How the model can be deployed to some type of robotic arm? #5

Closed Jensonzzs closed 6 months ago

Jensonzzs commented 6 months ago

Many thanks to the author for posting such a nice github project, I would like to ask how the model can be deployed to some type of robotic arm?

hyy0613 commented 6 months ago

Hi, I understand that you want to deploy this model to a real robot, and someone in our team did that, but I'm not responsible for that part of the details. In general, similar to the method mentioned in RT-2 paper, we will prepare a host as a server, which is responsible for running the model to generate discrete action, let's say the action space is 7-DoF($\Delta x$ , $\Delta y$ , $\Delta z$ , $\Delta \alpha$ , $\Delta \beta$ , $\Delta \gamma$ , gripper).Here involves the communication between the robot and the server, it needs to pass the image to the server, after which the generated 7-DoF action will be returned to the robot side. Different types of robot arms will basically have an interface for converting action to robot control signals.There are also some materials about that like "7-DoF control of a Franka Emika Panda robot arm".I'm also interested in it, so I think we can contact with each other and discuss the details.Hope this answer could help you.

Jensonzzs commented 6 months ago

Hi, I understand that you want to deploy this model to a real robot, and someone in our team did that, but I'm not responsible for that part of the details. In general, similar to the method mentioned in RT-2 paper, we will prepare a host as a server, which is responsible for running the model to generate discrete action, let's say the action space is 7-DoF(Δx , Δy , Δz , Δα , Δβ , Δγ , gripper).Here involves the communication between the robot and the server, it needs to pass the image to the server, after which the generated 7-DoF action will be returned to the robot side. Different types of robot arms will basically have an interface for converting action to robot control signals.There are also some materials about that like "7-DoF control of a Franka Emika Panda robot arm".I'm also interested in it, so I think we can contact with each other and discuss the details.Hope this answer could help you.

Thank you for your reply, it helps me a lot. Our team is not doing research in this area, but just had a need to use this knowledge, which is why I asked you. May I ask how to contact you? I'd like to ask you about this.

hyy0613 commented 6 months ago

You can contact me through the email I published at the bottom at README~