search

atenpas / gpd

Detect 6-DOF grasp poses in point clouds
BSD 2-Clause "Simplified" License
606 stars 233 forks source link

Is there a way to generate more grasps from the face of the object's point cloud #67

Open saikishor opened 5 years ago

saikishor commented 5 years ago

hello,

Is there a way to make gpd to generate more grasps towards the seen part of the point cloud?. Because I am unable to generate them in the follow case, and this is very trivial. Because having it to generate the candidates towards the cloud is trivial in most of the cases.

image

image image

Moreover, how does the gpd generate candidates for the objects that it can't even grasp?

atenpas commented 5 years ago

towards the seen part of the point cloud

Not sure what you mean exactly, but you can tell GPD where to look for grasps with these ROS messages: CloudIndexed or CloudSamples.

saikishor commented 5 years ago

Thank you, may be that's what I am looking for. I will take a closer look and get back to you. Thanks for the response.

saikishor commented 5 years ago

@atenpas How can we choose the appropriate indices corresponding to the available part of the object cloud. I am confused about setting the samples from the geometry_msgs::Point. Can you share how can it be done.

saikishor commented 5 years ago

@atenpas I see that in the tutorial2. you have given all the indices corresponding to the cup (which is the tabletop object in the scene), but in my case I am parsing directly the pointcloud of the object I want for the grasping, filtering the scene. In this case, does the results vary if I set the indices or not, or simply should I need to set all the indices of the cloud as samples?. Let me know.

saikishor commented 5 years ago

@atenpas can you please clarify on this?

atenpas commented 5 years ago

What do you mean with "available part of the object cloud"?

If you use the point cloud of the object, you don't need to use indices. However, you could use them if you wanted to grasp a specific object part, for example, the handle of a mug. Also, if you pass in only the object cloud, and your object is on some plane, for example, a table, the algorithm cannot check the grasps against collisions with the table. The preferred way for tabletop grasping is to pass in the whole point cloud, but sample only on the relevant parts (that is, the objects).

saikishor commented 5 years ago

@atenpas thanks for the response, so you mean If I pass the complete cloud and use the points of the clouds as the indices to search for the grasp, I mostly won't have the issue of the grasps colliding with the table?.

atenpas commented 5 years ago

What do you mean with "points of the clouds"?

saikishor commented 5 years ago

sorry, I meant the points of the object's cloud as indices. Will that solve the grasp collisions?

atenpas commented 5 years ago

Yes, that should do it.

saikishor commented 5 years ago

Thanks, I will check it and get back to you. Thanks for the confirmation.

saikishor commented 5 years ago

@atenpas I have given the below complete point cloud, and gave the all the points of one of the object as an input to the gpd but the gpd returned 0 grasp candidates. I have also attached the output of the gpd node for your reference. Am I doing anything wrong?

image

[ INFO] [1549377354.917781351, 11536.381000000]: Received cloud with 307200 points.
[ INFO] [1549377354.918074115, 11536.382000000]: Received object samples of 9152
[ INFO] [1549377354.918177561, 11536.382000000]: Table height in the grasp detector is set to 0.833346 m
Processing cloud with: 307200 points.
Calculating surface normals ...
Using integral images for surface normals estimation ...
 runtime (normals): 0.0829804
Reversing direction of normals that do not point to at least one camera ...
 reversed 94157 normals
 runtime (reverse normals): 0.00303335
9152 samples left after workspace filtering 
After workspace filtering: 126531 points left.
Using 70 out of 9152 available samples.
Subsampled 70 samples at random uniformly.
Estimating local reference frames ...
Estimated 0 local reference frames in 0.000725763 sec.
Finding hand poses ...
 Found 0 grasp candidate sets in 0.0033638 sec.
====> HAND SEARCH TIME: 0.0302814
[ INFO] [1549377355.040833529, 11536.488000000]: Generated 0 grasp candidate sets.
[ INFO] [1549377355.040870401, 11536.488000000]: Published 0 highest-scoring grasps.
[ INFO] [1549377364.872960024, 11545.707000000]: Received cloud with 307200 points.
[ INFO] [1549377364.873167139, 11545.708000000]: Received object samples of 9130
[ INFO] [1549377364.873200673, 11545.708000000]: Table height in the grasp detector is set to 0.833341 m
Processing cloud with: 307200 points.
Calculating surface normals ...
Using integral images for surface normals estimation ...
 runtime (normals): 0.0787009
Reversing direction of normals that do not point to at least one camera ...
 reversed 94157 normals
 runtime (reverse normals): 0.00239826
9130 samples left after workspace filtering 
After workspace filtering: 126531 points left.
Using 70 out of 9130 available samples.
Subsampled 70 samples at random uniformly.
Estimating local reference frames ...
Estimated 0 local reference frames in 0.000113611 sec.
Finding hand poses ...
 Found 0 grasp candidate sets in 0.00158908 sec.
====> HAND SEARCH TIME: 0.0324725
[ INFO] [1549377364.993061200, 11545.811000000]: Generated 0 grasp candidate sets.
[ INFO] [1549377364.993204764, 11545.811000000]: Published 0 highest-scoring grasps.
atenpas commented 5 years ago

From the text output, it's hard to tell. I'm suspicious about two outputs:

Using integral images for surface normals estimation ... Using 70 out of 9130 available samples.

You could try to increase the number of samples. More importantly though: what does it look like when you set plot_samples to true?

And what type of message do you use and what are the contents (on a high level)?

saikishor commented 5 years ago

@atenpas Thanks for the imminent response. I have tried as you said with more samples as high as 5000, but no success. I have also tried by setting the param plot_samples to true there is a popup screen but completely blank.

[ INFO] [1549382736.464853434, 11209.457000000]: Received cloud with 307200 points.
[ INFO] [1549382736.464955013, 11209.457000000]: Received object samples of 9135
Processing cloud with: 307200 points.
Calculating surface normals ...
Using integral images for surface normals estimation ...
 runtime (normals): 0.0929889
Reversing direction of normals that do not point to at least one camera ...
 reversed 88008 normals
 runtime (reverse normals): 0.00362407
9135 samples left after workspace filtering 
After workspace filtering: 126165 points left.
Using 5000 out of 9135 available samples.
Subsampled 5000 samples at random uniformly.
Estimating local reference frames ...
Estimated 0 local reference frames in 0.00459527 sec.
Finding hand poses ...
 Found 0 grasp candidate sets in 0.00203273 sec.

I am using below message type, where I will also be able to set the table height dynamically:

sensor_msgs/PointCloud2 pointcloud
geometry_msgs/Point[] samples
float32 table_height
atenpas commented 5 years ago

That "popup screen" is PCL's visualizer. You can press r to reset the camera to the origin, and h to see a list of commands. If the samples are correct, you should see something like this:

image

That does not look like a message from the GPD package. I suspect that is why it does not work. You should use the messages that are part of GPD (e.g., the one used in Tutorial 2).

saikishor commented 5 years ago

@atenpas Thanks I will check once again, I am doing the same what you do with your message data, in theory it shouldn't go wrong, but may be I might have mistaken at some place. I want to confirm one thing, I am using the branch forward not the master. Does this have something to do with the above problem?. I will update you.

Thank you.

saikishor commented 5 years ago

@atenpas You are right, I am able to visualise. I see that my points are not in the same frame so it was not generating earlier. I have one more question, sometimes the candidates are too far for the gripper to grasp as you see below. However, the grasps have been improved a lot with this procedure, thanks a lot for the timely responses.

grasping this cube, this is good as some of the candidates are well present to grasp the cube image

For this cylinder (COKE and PRINGLES), most of the time the grasp are bit far does it has something to do with the size of the object if I am not wrong?. image image image image

For the coke slim, it is again good: image image image

saikishor commented 5 years ago

@atenpas can you comment on the above results please?

atenpas commented 4 years ago

It looks like the coke/pringles cylinder in your second series of screenshots has a large diameter, i.e., it barely fits into the robot hand?

saikishor commented 4 years ago

@atenpas mayb for the pringles, but the coke fits good enough

atenpas commented 4 years ago

Upload the coke PCD file you're using and I can have a look.

qyp-robot commented 2 years ago

Hello, @atenpas may I ask, I am grabbing the object on the desktop. I completely segment the point cloud of the object, and then use GPD to capture and sample. I use a single camera, and I obtain part of the point cloud of the object; But I saw in the source code, about the angle calculation code between the grasping pose and the predefined vector, is the origin of this predefined vector located in the camera coordinate system, should some of the filtered poses avoid collision with the table?