what PR2 shoud do

[chiwunau]

1. publish first contact point and end contact point after every push
2. push 2 directions (if possible) , each direction is vertical to each other?
   • try not to break the arrangement of scenes.

[chiwunau]

What will happen if the push failed? like even though I thought the object has been moved but in fact it has not.

Will you give me a signal to try move the same object again OR every thing will just reset? (a new try)

[iKrishneel]

In case of push failure, the final step (object merging) that is after the push, will or may discover nothing changes and the the whole algorithm will repeat as usual.

Therefore we are not focused at detecting push failure. Everything will reset

[chiwunau]

I see, that makes life easier.

I am wondering do you have a preference of the moving order? like boxes on the bottom first, something like that.

sth notes of what I am thinking

1. position to left/right arm
2. size of box (move small one first, maybe)

moving methods a. grasp-and-move (if objects thin enough) b. push from top side and slide the object (if top surface big enough) c. push one of the vertical faces. b+c

[iKrishneel]

I think your choice of movements are ok. I dont necessarily have any order. My target is that the robot should move a object from the cluttered scene so we can separate the object.

Feel free to adapt your choice of movements.

[chiwunau]

rules choose two candidates for left/right arms

push from one of the sides of box

• should move away from centroid
• gripper won't collide objects
• no other objects on path.

if not satisfied

• able to pick?

if not satisfied push from top and slide

• away from centroid
• no collision on path

if not --> try other objects

[iKrishneel]

So these are the conditions for end-effector movements?

[chiwunau]

not final solution, just doing sth notes

how many cm should i move the object by default?

[iKrishneel]

Default 10cm should be ok

[chiwunau]

i build a prototype, the algorithm seems working , but PR2 sometimes can not solve inverse-kinematics.

[iKrishneel]

When it is not solved, can you detect that? If yes than can u select the next cluster?

[iKrishneel]

In the robot node, you will be doing the small volume bounding box filtering.

Can you add this to the above. I will provide you with the list of known bounding boxes together with small bounding boxes from the segmentation module in the first step.

1) Can you check the list of unknown bounding boxes within the region of known bounding boxes and also remove those.

2) After removing the unknown bounding boxes in (1) if the unknown bounding box list = 0 than just pass me a index of -1 in the topic "/robot_pushing_motion_node/pushed_box_index. Once I will see -1 I will assume that all is labeled and we stop processing.

Why I prefer you to do this part because you will be doing filtering so better to just put the two into once node.

[chiwunau]

surely I can try to do that, so remove unknown boxes from the pushing candidates those are

• A.Intersect with known boxes
• B. Inside known boxes

or only B?

[iKrishneel]

Maybe remove any boxes that intersect more than 50% .

[chiwunau]

OK

[chiwunau]

I will use all the condition you have suggested to filter the candidate boxes for pushing.. so let say if there is no any candidate i can pushing or all the available candidates are not able to be pushed(due to ik or collision), i will return -1.

also boxes that are not on table will always be marked as unknown. cos I will never push them.

[iKrishneel]

Yes thats correct

[iKrishneel]

Topic Name for known bounding box. "/manipulated_cluster/output/known_bounding_boxes"

[chiwunau]

how about unknown boxes?

[iKrishneel]

Unknow boxes will be passed from initial segmentation that is small clusters of boxes which are used for pushing.

Once known boxes are available you can filter the unknown boxes in known region

[chiwunau]

I mean the topic name

[iKrishneel]

Thats same topic u subscribe to initially that is from ueda san's algorithm

[chiwunau]

so i will get unknown boxes for only one time?

[chiwunau]

what I thought is that every time you get a known object, you will use your algorithm to Re-recognize the scene except for the known-box region. In that case I can get every box position updated.

There maybe a problem that In robot model, every time the roobot move the object and assume that it has been move exactly 10cm, but in real the object would not and the new boundingbox (known box) may not cover the region of the box in robot model. Also if the object are combined by like two boxes, In robot model only one box will be moved, and even though you reconstruct the two boxes in to one new box(known box) and pass it to be, the part I did think i have moved will not be filtered out.

[iKrishneel]

Actually, I thought to not segment the known object but I realized that the easiest way is just to give you the known boxes and the updated scene segmented region as set of unknown object boxes (here the known object will also be segmented into small boxes just like in the start)

In other words in each iteration, the object positions will be updated. Regardless if your push was 10cm or just 3cm, the segmentation in Node 1 will run to update the scene and will pass you all the updated boxes as unknown even those that was pushed. Node 3 will tell you known boxes from the last push.

You will need to filter this unknown boxes by size and those that intersects with the known boxes.

[iKrishneel]

Basically you need to subscribe to the known boxes topic and check the intersection and the volume conditions and remove these boxes, from the second iteration.

[chiwunau]

Also I have a very fundamental question, if you are using Ueda san program. I can also kind of solve the problem by keep pushing boxes(only one in beginning) from every direction until the bounding box won't seperate in to more small boxes with using only Ueda san program.

I think the advantages of your program is that (objects 1 is segmented in to boxA and boxB) (after pushing boxA, boxA and boxB move together) => (assume that boxA and box B are one object and make it box C) (by that you can feedback this result to the algorithm and improve it so that next time It will automaticly put boxA and boxB in to one box at the beginning)

[chiwunau]

so there would be two topic one for unknown boundingbox one for known boundingbox (empty at the beginning)

something like "/point_cloud_scene_decomposer_cluster_points/unknown_boxes" (segments of known & unknown objects) "/point_cloud_scene_decomposer_cluster_points/known_boxes" (segment of known objects)

and I update my scene every times using these two topics.

[iKrishneel]

Yes that is correct. Like you mentions boxA and boxB which is one object and if you push just boxA, the algorithm in Node 3 determines this movements of boxB and merges them into one boxC and pushes the data into known box topic.

However, the boxC is not passed to Node 1 to improve the result. Node 1 will always perform segmentation. Putting boxA and box B into one as box C is what I want you to do before determining which box to push. This will begin from the second iteration.

Use the topic name "/scene_decomposer/output/unknown_boxes" for unknown boxes.

[chiwunau]

OK, I just think Ueda san boundingbox should not be used (clustering by euclidean distance? I guess). and solve the problem (seperate the scene?) using solely things in your program, in order to claim that it is a new method.

[chiwunau]

close for humanoids published