search

octo-models / octo

Octo is a transformer-based robot policy trained on a diverse mix of 800k robot trajectories.
https://octo-models.github.io/
MIT License
861 stars 164 forks source link

What is the model number of the UR5 robotic arm, camera and claw? #72

Open KangriX opened 7 months ago

KangriX commented 7 months ago

I want to test the performance of the octo through online simulation and am about to use RoboDK to simulate the UR5 robotic arm. Or is there another easier way for me to test the performance of the octo? Thanks for the help!

KangriX commented 7 months ago

My purpose is to test the performance of octo as I am going to use model compression technique and I want to compare the performance of octo before and after compression. On-line simulation is one of the ways I've found so far, but I don't know the model of the robotic arm used in the experiment and the model of its tooling, its mounting location, etc. This is a very cumbersome task, so if there are other simpler ways to test the performance, please let me know immediately! Many thanks to everyone for their help!

kpertsch commented 7 months ago

The best way to test Octo's performance is on a real robot system, eg the Bridge V2 system, which is fairly low-cost. If you want to work in simulation, it is unlikely that the model will work out-of-the-box and you'll likely need to collect a few demonstrations in your sim environment and fine-tune the model, but then you could evaluate your model compression techniques on that fine-tuned version of Octo.

Leeviber commented 6 months ago

Also have question about camera mounting location. If I want to test the model performance out-of-box on real UR5 robtic arm, do I need set up the observation space of UR5 exactly matches the observations in the dataset, such as camera position, arm mount position.

For example the dataset berkeley_autolab_ur5, the main camera image as shown in beleow, the camera is on the side of the arm, if I want change the camera position to the top of the arm or the arm will mount at a 45-degree angle, is that OK? Does it will impact the model performance a lot? Screenshot from 2024-04-16 17-14-37

If not OK, what kind of rule shoule follow for fine-tuning dataset? Specifically, how should the images include the coverage of the arm? What is the ideal setup for this scenario?

Nimingez commented 5 months ago

Have you ever figure out？ @Leeviber