Rofunc: The Full Process Python Package for Robot Learning from Demonstration and Robot Manipulation
Repository address: https://github.com/Skylark0924/Rofunc
Documentation: https://rofunc.readthedocs.io/
Rofunc package focuses on the Imitation Learning (IL), Reinforcement Learning (RL) and Learning from Demonstration (LfD) for (Humanoid) Robot Manipulation. It provides valuable and convenient python functions, including
demonstration collection, data pre-processing, LfD algorithms, planning, and control methods. We also provide an
IsaacGym
and OmniIsaacGym
based robot simulator for evaluation. This package aims to advance the field by building a full-process
toolkit and validation platform that simplifies and standardizes the process of demonstration data collection,
processing, learning, and its deployment on robots.
Citation
If you use rofunc in a scientific publication, we would appreciate citations to the following paper:
@software{liu2023rofunc,ย
title = {Rofunc: The Full Process Python Package for Robot Learning from Demonstration and Robot Manipulation},
author = {Liu, Junjia and Dong, Zhipeng and Li, Chenzui and Li, Zhihao and Yu, Minghao and Delehelle, Donatien and Chen, Fei},
year = {2023},
publisher = {Zenodo},
doi = {10.5281/zenodo.10016946},
url = {https://doi.org/10.5281/zenodo.10016946},
dimensions = {true},
google_scholar_id = {0EnyYjriUFMC},
}
[!WARNING]
If our code is found to be used in a published paper without proper citation, we reserve the right to address this issue formally by contacting the editor to report potential academic misconduct!
ๅฆๆๆไปฌ็ไปฃ็ ่ขซๅ็ฐ็จไบๅทฒๅ่กจ็่ฎบๆ่ๆฒกๆ่ขซๆฐๅฝๅผ็จ๏ผๆไปฌไฟ็้่ฟๆญฃๅผ่็ณป็ผ่พๆฅๅๆฝๅจๅญฆๆฏไธ็ซฏ่กไธบ็ๆๅฉใ
Update News ๐๐๐
v0.0.2.6 Support dexterous grasping and human-humanoid robot skill transfer
- [2024-06-30] ๐๐ Human-level skill transfer from human to heterogeneous humanoid robots have been completed and are awaiting release.
- [2024-01-24] ๐ CURI Synergy-based Softhand grasping tasks are supported to be trained by
RofuncRL
.
- [2023-12-03] ๐ผ๏ธ Segment-Anything (SAM) is supported in an interactive mode, check the examples in Visualab (segment anything, segment with prompt).
- [2023-10-31] ๐
RofuncRL
: A modular easy-to-use Reinforcement Learning sub-package designed for Robot Learning tasks is released. It has been tested with simulators like OpenAIGym
, IsaacGym
, OmniIsaacGym
(see example gallery), and also differentiable simulators like PlasticineLab
and DiffCloth
.
- ...
- If you want to know more about the update news, please refer to the changelog.
Installation
Please refer to the installation guide.
Documentation
To give you a quick overview of the pipeline of rofunc
, we provide an interesting example of learning to play Taichi
from human demonstration. You can find it in the Quick start
section of the documentation.
The available functions and plans can be found as follows.
> **Note**
> โ
: Achieved ๐: Reformatting โ: TODO
| Data | | Learning | | P&C | | Tools | | Simulator | |
|:-------------------------------------------------------------------------------------------------------:|---|:------------------------------------------------------------------------------------------------------:|----|:------------------------------------------------------------------------------------------------------------------:|----|:-------------------------------------------------------------------------------------------------------------------:|---|:------------------------------------------------------------------------------------------------------------:|----|
| [`xsens.record`](https://rofunc.readthedocs.io/en/latest/devices/xsens.html) | โ
| `DMP` | โ | [`LQT`](https://rofunc.readthedocs.io/en/latest/planning/lqt.html) | โ
| `config` | โ
| [`Franka`](https://rofunc.readthedocs.io/en/latest/simulator/franka.html) | โ
|
| [`xsens.export`](https://rofunc.readthedocs.io/en/latest/devices/xsens.html) | โ
| [`GMR`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.gmr.html) | โ
| [`LQTBi`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqt.lqt.html) | โ
| [`logger`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.logger.beauty_logger.html) | โ
| [`CURI`](https://rofunc.readthedocs.io/en/latest/simulator/curi.html) | โ
|
| [`xsens.visual`](https://rofunc.readthedocs.io/en/latest/devices/xsens.html) | โ
| [`TPGMM`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmm.html) | โ
| [`LQTFb`](https://rofunc.readthedocs.io/en/latest/planning/lqt_fb.html) | โ
| [`datalab`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.datalab.html) | โ
| `CURIMini` | ๐ |
| [`opti.record`](https://rofunc.readthedocs.io/en/latest/devices/optitrack.html) | โ
| [`TPGMMBi`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmm.html) | โ
| [`LQTCP`](https://rofunc.readthedocs.io/en/latest/planning/lqt_cp.html) | โ
| [`robolab.coord`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.robolab.coord.transform.html) | โ
| [`CURISoftHand`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.simulator.curi_sim.html) | โ
|
| [`opti.export`](https://rofunc.readthedocs.io/en/latest/devices/optitrack.html) | โ
| [`TPGMM_RPCtl`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmm.html) | โ
| [`LQTCPDMP`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqt.lqt_cp_dmp.html) | โ
| [`robolab.fk`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.robolab.kinematics.fk.html) | โ
| [`Walker`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.simulator.walker_sim.html) | โ
|
| [`opti.visual`](https://rofunc.readthedocs.io/en/latest/devices/optitrack.html) | โ
| [`TPGMM_RPRepr`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmm.html) | โ
| [`LQR`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.lqr.html) | โ
| [`robolab.ik`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.robolab.kinematics.ik.html) | โ
| `Gluon` | ๐ |
| [`zed.record`](https://rofunc.readthedocs.io/en/latest/devices/zed.html) | โ
| [`TPGMR`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmr.html) | โ
| [`PoGLQRBi`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.lqr.html) | โ
| `robolab.fd` | โ | `Baxter` | ๐ |
| [`zed.export`](https://rofunc.readthedocs.io/en/latest/devices/zed.html) | โ
| [`TPGMRBi`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tpgmr.html) | โ
| [`iLQR`](https://rofunc.readthedocs.io/en/latest/planning/ilqr.html) | ๐ | `robolab.id` | โ | `Sawyer` | ๐ |
| [`zed.visual`](https://rofunc.readthedocs.io/en/latest/devices/zed.html) | โ
| [`TPHSMM`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.learning.ml.tphsmm.html) | โ
| [`iLQRBi`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.ilqr_bi.html) | ๐ | [`visualab.dist`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.visualab.distribution.html) | โ
| [`Humanoid`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.simulator.humanoid_sim.html) | โ
|
| [`emg.record`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.devices.emg.record.html) | โ
| [`RLBaseLine(SKRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RLBaseLine/SKRL.html) | โ
| `iLQRFb` | ๐ | [`visualab.ellip`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.visualab.ellipsoid.html) | โ
| [`Multi-Robot`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.simulator.multirobot_sim.html) | โ
|
| [`emg.export`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.devices.emg.export.html) | โ
| `RLBaseLine(RLlib)` | โ
| [`iLQRCP`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.ilqr_cp.html) | ๐ | [`visualab.traj`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.visualab.trajectory.html) | โ
| | |
| `mmodal.record` | โ | `RLBaseLine(ElegRL)` | โ
| [`iLQRDyna`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.ilqr_dyna.html) | ๐ | [`oslab.dir_proc`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.oslab.dir_process.html) | โ
| | |
| [`mmodal.sync`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.devices.mmodal.sync.html) | โ
| `BCO(RofuncIL)` | ๐ | [`iLQRObs`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.planning_control.lqr.ilqr_obstacle.html) | ๐ | [`oslab.file_proc`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.oslab.file_process.html) | โ
| | |
| | | `BC-Z(RofuncIL)` | โ | `MPC` | โ | [`oslab.internet`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.oslab.internet.html) | โ
| | |
| | | `STrans(RofuncIL)` | โ | `RMP` | โ | [`oslab.path`](https://rofunc.readthedocs.io/en/latest/apidocs/rofunc/rofunc.utils.oslab.path.html) | โ
| | |
| | | `RT-1(RofuncIL)` | โ | | | | | | |
| | | [`A2C(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/A2C.html) | โ
| | | | | | |
| | | [`PPO(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/PPO.html) | โ
| | | | | | |
| | | [`SAC(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/SAC.html) | โ
| | | | | | |
| | | [`TD3(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/TD3.html) | โ
| | | | | | |
| | | `CQL(RofuncRL)` | โ | | | | | | |
| | | `TD3BC(RofuncRL)` | โ | | | | | | |
| | | [`DTrans(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/DTrans.html) | โ
| | | | | | |
| | | `EDAC(RofuncRL)` | โ | | | | | | |
| | | [`AMP(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/AMP.html) | โ
| | | | | | |
| | | [`ASE(RofuncRL)`](https://rofunc.readthedocs.io/en/latest/lfd/RofuncRL/ASE.html) | โ
| | | | | | |
| | | `ODTrans(RofuncRL)` | โ | | | | | | |
RofuncRL
RofuncRL
is one of the most important sub-packages of Rofunc
. It is a modular easy-to-use Reinforcement Learning sub-package designed for Robot Learning tasks. It has been tested with simulators like OpenAIGym
, IsaacGym
, OmniIsaacGym
(see example gallery), and also differentiable simulators like PlasticineLab
and DiffCloth
. Here is a list of robot tasks trained by RofuncRL
:
Note\
You can customize your own project based on RofuncRL by following the RofuncRL customize tutorial.\
We also provide a RofuncRL-based repository template to generate your own repository following the RofuncRL structure by one click.\
For more details, please check the documentation for RofuncRL.
The list of all supported tasks.
| Tasks | Animation | Performance | [ModelZoo](https://github.com/Skylark0924/Rofunc/blob/main/rofunc/config/learning/model_zoo.json) |
|-------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------|---------------------------------------------------------------------------------------------------|
| Ant | ![](doc/img/task_gifs/AntRofuncRLPPO.gif) | | โ
|
| Cartpole | | | |
| Franka
Cabinet | ![](doc/img/task_gifs/FrankaCabinetRofuncRLPPO.gif) | | โ
|
| Franka
CubeStack | | | |
| CURI
Cabinet | ![](doc/img/task_gifs/CURICabinetRofuncRLPPO.gif) | | โ
|
| CURI
CabinetImage | ![](doc/img/task_gifs/CURICabinetRofuncRLPPO.gif) | | |
| CURI
CabinetBimanual | | | |
| CURIQbSoftHand
SynergyGrasp | | | โ
|
| Humanoid | ![](doc/img/task_gifs/HumanoidRofuncRLPPO.gif) | | โ
|
| HumanoidAMP
Backflip | ![](doc/img/task_gifs/HumanoidFlipRofuncRLAMP.gif) | | โ
|
| HumanoidAMP
Walk | | | โ
|
| HumanoidAMP
Run | ![](doc/img/task_gifs/HumanoidRunRofuncRLAMP.gif) | | โ
|
| HumanoidAMP
Dance | ![](doc/img/task_gifs/HumanoidDanceRofuncRLAMP.gif) | | โ
|
| HumanoidAMP
Hop | ![](doc/img/task_gifs/HumanoidHopRofuncRLAMP.gif) | | โ
|
| HumanoidASE
GetupSwordShield | ![](doc/img/task_gifs/HumanoidASEGetupSwordShieldRofuncRLASE.gif) | | โ
|
| HumanoidASE
PerturbSwordShield | ![](doc/img/task_gifs/HumanoidASEPerturbSwordShieldRofuncRLASE.gif) | | โ
|
| HumanoidASE
HeadingSwordShield | ![](doc/img/task_gifs/HumanoidASEHeadingSwordShieldRofuncRLASE.gif) | | โ
|
| HumanoidASE
LocationSwordShield | ![](doc/img/task_gifs/HumanoidASELocationSwordShieldRofuncRLASE.gif) | | โ
|
| HumanoidASE
ReachSwordShield | | | โ
|
| HumanoidASE
StrikeSwordShield | ![](doc/img/task_gifs/HumanoidASEStrikeSwordShieldRofuncRLASE.gif) | | โ
|
| BiShadowHand
BlockStack | ![](doc/img/task_gifs/BiShadowHandBlockStackRofuncRLPPO.gif) | | โ
|
| BiShadowHand
BottleCap | ![](doc/img/task_gifs/BiShadowHandBottleCapRofuncRLPPO.gif) | | โ
|
| BiShadowHand
CatchAbreast | ![](doc/img/task_gifs/BiShadowHandCatchAbreastRofuncRLPPO.gif) | | โ
|
| BiShadowHand
CatchOver2Underarm | ![](doc/img/task_gifs/BiShadowHandCatchOver2UnderarmRofuncRLPPO.gif) | | โ
|
| BiShadowHand
CatchUnderarm | ![](doc/img/task_gifs/BiShadowHandCatchUnderarmRofuncRLPPO.gif) | | โ
|
| BiShadowHand
DoorOpenInward | ![](doc/img/task_gifs/BiShadowHandDoorOpenInwardRofuncRLPPO.gif) | | โ
|
| BiShadowHand
DoorOpenOutward | ![](doc/img/task_gifs/BiShadowHandDoorOpenOutwardRofuncRLPPO.gif) | | โ
|
| BiShadowHand
DoorCloseInward | ![](doc/img/task_gifs/BiShadowHandDoorCloseInwardRofuncRLPPO.gif) | | โ
|
| BiShadowHand
DoorCloseOutward | ![](doc/img/task_gifs/BiShadowHandDoorCloseOutwardRofuncRLPPO.gif) | | โ
|
| BiShadowHand
GraspAndPlace | ![](doc/img/task_gifs/BiShadowHandGraspAndPlaceRofuncRLPPO.gif) | | โ
|
| BiShadowHand
LiftUnderarm | ![](doc/img/task_gifs/BiShadowHandLiftUnderarmRofuncRLPPO.gif) | | โ
|
| BiShadowHand
HandOver | ![](doc/img/task_gifs/BiShadowHandOverRofuncRLPPO.gif) | | โ
|
| BiShadowHand
Pen | ![](doc/img/task_gifs/BiShadowHandPenRofuncRLPPO.gif) | | โ
|
| BiShadowHand
PointCloud | | | |
| BiShadowHand
PushBlock | ![](doc/img/task_gifs/BiShadowHandPushBlockRofuncRLPPO.gif) | | โ
|
| BiShadowHand
ReOrientation | ![](doc/img/task_gifs/BiShadowHandReOrientationRofuncRLPPO.gif) | | โ
|
| BiShadowHand
Scissors | ![](doc/img/task_gifs/BiShadowHandScissorsRofuncRLPPO.gif) | | โ
|
| BiShadowHand
SwingCup | ![](doc/img/task_gifs/BiShadowHandSwingCupRofuncRLPPO.gif) | | โ
|
| BiShadowHand
Switch | ![](doc/img/task_gifs/BiShadowHandSwitchRofuncRLPPO.gif) | | โ
|
| BiShadowHand
TwoCatchUnderarm | ![](doc/img/task_gifs/BiShadowHandTwoCatchUnderarmRofuncRLPPO.gif) | | โ
|
Star History
Related Papersย ย ย ย ย ย ย ย ย ย
- Robot cooking with stir-fry: Bimanual non-prehensile manipulation of semi-fluid objects (IEEE RA-L 2022 | Code)
@article{liu2022robot,
title={Robot cooking with stir-fry: Bimanual non-prehensile manipulation of semi-fluid objects},
author={Liu, Junjia and Chen, Yiting and Dong, Zhipeng and Wang, Shixiong and Calinon, Sylvain and Li, Miao and Chen, Fei},
journal={IEEE Robotics and Automation Letters},
volume={7},
number={2},
pages={5159--5166},
year={2022},
publisher={IEEE}
}
- SoftGPT: Learn Goal-oriented Soft Object Manipulation Skills by Generative Pre-trained Heterogeneous Graph
Transformer (IROS 2023๏ฝCode coming soon)
@inproceedings{liu2023softgpt,
title={Softgpt: Learn goal-oriented soft object manipulation skills by generative pre-trained heterogeneous graph transformer},
author={Liu, Junjia and Li, Zhihao and Lin, Wanyu and Calinon, Sylvain and Tan, Kay Chen and Chen, Fei},
booktitle={2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
pages={4920--4925},
year={2023},
organization={IEEE}
}
- BiRP: Learning Robot Generalized Bimanual Coordination using Relative Parameterization Method on Human
Demonstration (IEEE CDC 2023 | Code)
@inproceedings{liu2023birp,
title={Birp: Learning robot generalized bimanual coordination using relative parameterization method on human demonstration},
author={Liu, Junjia and Sim, Hengyi and Li, Chenzui and Tan, Kay Chen and Chen, Fei},
booktitle={2023 62nd IEEE Conference on Decision and Control (CDC)},
pages={8300--8305},
year={2023},
organization={IEEE}
}
The Team
Rofunc is developed and maintained by
the CLOVER Lab (Collaborative and Versatile Robots Laboratory), CUHK.
Acknowledge
We would like to acknowledge the following projects:
Learning from Demonstration
- pbdlib
- Ray RLlib
- ElegantRL
- SKRL
- DexterousHands
Planning and Control
- Robotics codes from scratch (RCFS)