search

hjalte33 / rl_unscrew

A Reinforcement Learning package for 6th semester project
11 stars 10 forks source link

Continuous action space and state space #1

Open murtazabasu opened 4 years ago

murtazabasu commented 4 years ago

Hello, I am currently working on a project which involves stimulating and coordinating two UR5 in ROS/Gazebo for a pick and place task i.e. (the first robot picking up an object from a given position and moving to a certain position and the second robot moving to that position, taking the object from the first robot and going to the goal position). I am using Robotiq85 grippers for picking up an object till now I am successfully able to perform this task by hard-coding it, you can find the video here (https://www.youtube.com/watch?v=n6Vk9lIxKkg) but I want to perform this task using PPO for which I need to create an environment such that when the agent takes an action that actions is performed into the simulated world using Moveit Python interface (library which is used to control motions to the robot in Gazebo-simulated world). Your work seems to be a bit promising for making a baseline for my project but my question is since you used discrete states and actions the moveit was able to perform the simulation but in my case where I have continuous states and actions will moveit commander interface be fast to simulate the actions in Gazebo?

hjalte33 commented 4 years ago

Just as a disclaimer, I made this project quite a while ago and its not maintained at the moment so i'm quite rusty on the topic, but i'll be happy to help with what i can. In my experience MoveIt has trouble executing the movement of two robots at the same time and it'll require a specific setup of moveit in order to do so which i'm not sure how to do. Else you are limited to move one robot at a time. The discretization of the movement is only to do our reinforcement learning, and i think MoveIt can do it continuously as well. I can maybe point you to a more related project done at my university where they actually used dual arms and moveit together. I don't know if you can find the article. I'm not sure the repo is publicly available, but if interested, i can maybe put you in contact with some of the people involved.

A Dual-arm Collaborative Robot System for the Smart Factories of the Future. / Filtenborg Buhl, Jens; Grønhøj, Rune; Kjær Jørgensen, Jan; Mateus, Guilherme; Pinto, Daniela; Krunderup Sørensen, Jacob; Bøgh, Simon; Chrysostomou, Dimitrios. 29th International Conference on Flexible Automation and Intelligent Manufacturing: FAIM 2019. 2019.

murtazabasu commented 4 years ago

@hjalte33 Thank you for replying, it would be great if you can put me up in contact that would be a great help!! Thanks again

hjalte33 commented 4 years ago

If you contact Dimitris from the article i send you. He's the supervisor for the student group working with our Dual robot setup. I talked with him and he'll help you to get in touch with them. https://vbn.aau.dk/en/persons/130697

murtazabasu commented 4 years ago

@hjalte33 Thank you for the info by the way I mailed Mr. Dimitris yesterday but I haven't heard of him till now. I have made my agent and it is learning in the environment the only problem is collision check. The UR5 arm gets collided to the surface and objects which disrupts the entire simulation. Do you by chance develop any contact sensor for the UR5 which gives ROS message when the end effector is close to some object?

hjalte33 commented 4 years ago

Give him some time, he's a busy man. Collisions are always a trouble to deal with in Gazebo. I don't know if gazebo has a build in collision detection engine so you don't have to write your own. I think it does but i'm not sure how it then would work. Sorry can't help much more .