search

BarisYazici / deep-rl-grasping

Train deep reinforcement learning model for robotics grasping. Choose from different perception layers raw Depth, RGBD and autoencoder. Test the learned models in different scenes and object datasets
MIT License
192 stars 44 forks source link

Hi!You have a great job,could I ask you some questions? #2

Closed jamesking23 closed 3 years ago

jamesking23 commented 3 years ago

I'm interested in your work,how to use encoder to train SAC?What's the input of DRL algorithm?And which is the way to use your work to train Kuka envrionment?

BarisYazici commented 3 years ago

For Encoder

There is a config file under the config folder which is called gripper_grasp.yaml, you can modify the file to change perception type, curriculum parameters, or SAC parameters This file you need to change this line to False. If depth and full observation are both set to false, the autoencoder perception will be used by default. To train with SAC you need to simply pass --algo SAC argument when you start the training. For example python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir SAC_auto_ecoder --timestep 100000 -v This line will simply start a new training with SAC algorithm 100k steps using the gripper_grasp.yaml configuration parameters. You can omit the final -v which only starts the PyBullet GUI.

Input of DRL Algorithm

DRL algorithm takes the state observation and outputs the actions which maximize the expected reward. Notice that state observation changes with the setting you set in gripper_grasp.yaml. This is where we set the observation space. Here we set the camera based on the perception type you enter in gripper_grasp.yaml.

Kuka Robot Training

I recently pushed the Kuka environment code to the KukaExt branch. SImply do git checkout kukaExt under the repository folder. There you can play with the Kuka Robot environment. Notice that now config file is different it takes different robot path. So now you can try to run pre-trained models by python manipulation_main/training/train_stable_baselines.py run --model trained_models/table_clearing/SAC_real_2m_buffer_128/best_model/best_model.zip -v or fine-tune a pre-trained model with python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir test_kuka_train_load --load_dir trained_models/table_clearing/SAC_real_2m_buffer_128/best_model/best_model.zip -v

or simply train the Kuka robot from zero

python manipulation_main/training/train_stable_baselines.py train --config config/gripper_grasp.yaml --algo SAC --model_dir kuka_train --timesteps 1000000

Hope this helps :)

jamesking23 commented 3 years ago

Thanks for your help!