Advice for running on other robot ?

[apirrone]

Hi @MarkFzp @tonyzhaozh,

I managed to run yout code and train a policy from sim_transfer_cube_scripted (It is not really working, but it just probably need some more training steps)

I would like to run mobile-aloha on a different mobile bimanual robot (Reachy https://www.pollen-robotics.com/)

Could you give me an overview of the required steps ?

Do I need to describe the new robot in the mujoco format ? I am not sure I understand properly what is the role of the simulated environment in your training pipeline. Reading the paper, I did not really understant how you would specify a task like "Wipe wine" in the simulated environment.

Anyways, I guess I need to record episodes of a teleoperated task, and format it like in https://github.com/MarkFzp/mobile-aloha/blob/main/aloha_scripts/record_episodes.py ?

Thank you very much for your help !

Antoine

[apirrone]

I was able to adapt the code to train and run with my robot :)

I can train policies from scratch based on recordings of teleoperation sessions. I haven't trained it for a long time yet but it seems promising !

Now I would like to perform "co-training" as described in the paper. Where can I download the dataset ? And how do I use it along with a different robot architecture ?

Thanks !

[youliangtan]

I also briefly dig into the code, just to comment here if people would like to know the answer.

• to run on another robot, user needs to create a custom robot env. Refer to here. In this case this script uses aloha_scripts from the mobile aloha repo.
• I might be wrong, the sim env mainly works as a CI or demo example to run the pipeline without the real robot. It uses a scripted policy to generate a traj dataset, then it showcases a simple BC training.
• Refered to the paper, cotraining trains along with the static aloha arms dataset. The dataset has already been provided, mentioned on the website.

[tonyzhaozh]

Thanks for the response @youliangtan , you are absolutely correct.

The sim envs are just legacy envs from the original ACT code (https://github.com/tonyzhaozh/act). They do not have the mobile component and we are not using them actively in the mobile-aloha project.

[uuu686]

@tonyzhaozh hello, I also want to know, if my robot has different architecture(such as arms), can I use the co-training with the static ALOHA data? Thanks

[zxfever]

@apirrone hi bro, could you give some details to run the code on my own robot,thanks a lot.