search

incognite-lab / myGym

myGym enables fast prototyping of RL in the area of robotic manipulation and navigation.You can train different robots, in several environments on various tasks. There is automatic evaluation and benchmark tool. From version 2.1 there is support for multi-step tasks, multi-reward training and multi-network architectures.
https://mygym.readthedocs.io/en/latest/
MIT License
49 stars 11 forks source link

about "pick and place" task #33

Closed TheQuyen97ht closed 4 months ago

TheQuyen97ht commented 1 year ago

First of all, i want to thank all of you for this amazing work. I'm new to RL in Robotics and I want to make my own custom config files ( my task goal is picking and placing obj with the mechanical-gripper, not the magnetic one). There's something that confuses me when picking the right "observation" . Here's my config files: {

Environment

"env_name"  :"Gym-v0",
"workspace" :"table",
"engine"    :"pybullet",
"render"    :"opengl",
"camera"    :0,
"gui"       :0,
"visualize" :0,
"visgym"    :0,
#Robot
"robot"         :"panda",
"robot_action"  :"joints_gripper",
"robot_init"    :[-0.4, 0.4, 0.4],
"max_velocity"  :1,
"max_force"     :70,
"action_repeat" :1,
#Task
"task_type"             :"pnp",
"task_objects"          :[{"init":{"obj_name":"cube_holes","fixed":0,"rand_rot":0, "sampling_area":[-0.38,-0.38,0.42,0.42,0.1,0.1]},
                       "goal":{"obj_name":"cube_target","fixed":1,"rand_rot":0, "sampling_area":[0.3,-0.2,0.4,0.7,0.15,0.15]}}],
"color_dict"            : {"cube_holes":[[0.2,0.5,0.2,1]], "target":[[0.3,0.3,0.3,1]]},
"used_objects"          :{"num_range":[0,0], "obj_list":[]},
// Observation
// actual_state options (pick one): "endeff_xyz", "endeff_6D" (robot end effector), "obj_xyz", "obj_6D", "vae", "yolact", "voxel" or "dope"
// goal_state options (pick one): "obj_xyz", "obj_6D", "vae", "yolact", "voxel" or "dope"
// additional_obs options (pick none/one/more): "joints_xyz", "joints_angles", "endeff_xyz", "endeff_6D", "touch", "distractor"
"observation"           : {"actual_state":"obj_xyz", "goal_state":"obj_xyz", "additional_obs":["endeff_xyz", "touch"]},
#Distractor
"distractors"           : {"list":null, "moveable":1, "constant_speed":0, "movement_dims":3,
                                   "movement_endpoints":[-0.3, 0.3, 0.4, 0.7, 0.1, 0.3]},
#Reward
"reward"        :"pnp",
"distance_type" :"euclidean",
"vae_path"      :null,
"yolact_path"   :null,
"yolact_config" :null,
#Train
"train_framework"   :"tensorflow",
"algo"              :"ppo2",
"num_networks"      :1,
"max_episode_steps" :512,
"algo_steps"        :512,
"steps"             :10000000,
"pretrained_model"  :null,
"multiprocessing"   :false,
#Evaluation
"eval_freq"     :500000,
"eval_episodes" :100,
#Saving and logging
"logdir"    :"trained_models/my_pnp1n",
"record"    :0
}

After training, the results seem to be bad with a very low Pick-rate. Which details can i change/edit to suit my goal best in the config files ? Thank you

michalvavrecka commented 1 year ago

hi, we are on vacation AFK, so I am just guessing.

  1. Try different robot in config - see helpers.py for robot with gripper
  2. Change force and speed in config to 50 and 1 otherwise real picking is very hard as robot is too strong and fast
  3. change friction of the picked object in urdf to increase gripper success (simulated physics simplifies contact points)
  4. Try to use box gripper for panda

We are now progressing with multistep tasks so real gripping was not updated. Look at old commits and watch for gripper in it and test old configs dedicated to real gripping

You can also add touch sensors to robot urdf and add touch to observation, it will improve learning