BarisYazici / deep-rl-grasping

Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets
MIT License
187 stars 44 forks source link
deep-learning kuka machine-learning open-ai pybullet reinfrocement-learning robotics

Deep Reinforcement Learning on Robotics Grasping

Train robotics model with integrated curriculum learning-based gripper environment. Choose from different perception layers depth, RGB-D. Run pretrained models with SAC, BDQ and DQN algorithms. Test trained algorithms in different scenes and domains.

Master's thesis PDF

Prerequisites (CPU)

Install anaconda. Start a clean conda environment.

conda create -n grasp_env python=3.6
conda activate grasp_env

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Prerequisites (GPU)

conda create -n grasp_env python=3.6
conda activate grasp_env
conda install -c conda-forge cudatoolkit=10.0 cudnn=7.6.5
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$CONDA_PREFIX/lib/

In setup.py change:

'tensorflow==1.14.0',
to
'tensorflow_gpu==1.14.0',

Installation

Use pip to install the dependencies.

pip install -e .

If using GPU you can check if it was successful with: ´´´ python -c "import tensorflow as tf; print(tf.config.experimental.list_physical_devices('GPU'))" ´´´

Run Models

train_stable_baselines script provides the functionality of running and training models.

For running models 'manipulation_main/training/train_stable_baselines.py' takes the following arguments

For running functionality run sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py run --model trained_models/SAC_full_depth_1mbuffer/best_model/best_model.zip -v -t

Train models

For training models 'manipulation_main/training/train_stable_baselines.py' takes the following arguments

For training functionality train sub-parser needs to be passed to the script.

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir trained_models/SAC_full --timestep 100000 -v

Running the tests

To run the gripperEnv related test use

pytest tests_gripper

Authors

Citing the Project

To cite the master's thesis:

@MastersThesis{Yazici2020,
    author     =     {Yazici Baris},
    title     =     {{Branch Dueling Deep Q-Networks for Robotics Applications}},
    school     =     {Technical University of Munich},
    year     =     {2020},
    howpublished = {\url{https://github.com/BarisYazici/tum_masters_thesis}}
}

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Acknowledgments