search

Genesis-Embodied-AI / RoboGen

A generative and self-guided robotic agent that endlessly propose and master new skills.
Apache License 2.0
545 stars 49 forks source link

Support for Dual-Arm Robot URDFs and Automatic Task Generation in RoboGen #28

Closed return-sleep closed 1 week ago

return-sleep commented 1 month ago

Hi~

I am exploring the possibility of integrating dual-arm robot configurations. Specifically, I am interested in whether it is possible to import URDF files for dual-arm robots and use RoboGen to automatically generate tasks and demonstrations? Could you provide guidance on how to achieve this with RoboGen?

yufeiwang63 commented 1 month ago

Hi,

Thanks for your interest in our project.

It should be easy to import two robotic arms; I think you just need to import two arms in sim.py. These are the lines where we load the robot: https://github.com/Xingyu-Lin/softagent_rpad/blob/navin_dev/dressing_motion/curl/train_perturb.py#L257-L259. I think you just need to load another arm here.

For generating tasks with dual arms, you might need to change the task proposal prompt here: https://github.com/Genesis-Embodied-AI/RoboGen/blob/main/gpt_4/prompts/prompt_manipulation.py#L10. You can modify the prompt to tell GPT-4 that there are two arms in the scene.

For demonstration generation: For the motion planning part, you probably need to modify the motion planning utils such that it can plan path for two robots: https://github.com/Genesis-Embodied-AI/RoboGen/blob/main/manipulation/motion_planning_utils.py#L8-L18. I think just adding another instance of PbOMPLRobot for the second robotic arm should work. For the RL part, I think you probably need to modify the observation and action space to support both arms. The observation and action function are defined here in sim.py: https://github.com/Genesis-Embodied-AI/RoboGen/blob/main/manipulation/sim.py#L910, and https://github.com/Genesis-Embodied-AI/RoboGen/blob/main/manipulation/sim.py#L750