search

RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.35k stars 1.27k forks source link

Integrate grasp planner into Drake #3178

Closed hongkai-dai closed 5 years ago

hongkai-dai commented 8 years ago

We want to plan the grasping posture of the robot hand. We can try the following three approaches

  1. Integrate GraspIt! http://graspit-simulator.github.io/ into Drake.
  2. Write our own sampling based grasping toolkit.
  3. Implement the optimization-based grasping planner in C++ (We have one of them implemented in MATLAB side of Drake).

I think the first approach (GraspIt!) seems most viable. I will start with that.

Once we have one grasp planner in Drake, we should be able to evaluate how robust it is using IIWA arm (in the position control mode), together with our gripper (Schunk hand or the Robotiq hand).

liangfok commented 8 years ago

SGTM. I recall GraspIt! adopting an unusual (to me) coordinate frame convention, especially w.r.t. the world. You might want to check that first to ensure we don't start off in the wrong direction.

rpoyner-tri commented 8 years ago

Looks to me like the GraspIt! license is GPL. Are we allowed to integrate viral-license software with Drake?

liangfok commented 8 years ago

The following web page indicates that as long as Drake and GraspIt! are decoupled by a middleware, Drake does not need to adopt GPL.

http://programmers.stackexchange.com/questions/110380/call-gpl-software-from-non-gpl-software

jwnimmer-tri commented 8 years ago

(a) I don't think the page says that anywhere and (b) the mere fact of middleware is not enough to escape the GPL; the license can still infect intimately coupled software, no matter the mechanism.

liangfok commented 8 years ago

Here's the quote I read from the above-linked webpage:

You can use a GPLed program from your own program without your program being affected by the GPL, but you cannot link the GPLed code into your own program without your program becoming subject to the GPL's terms.

If there's a middleware like LCM or ROS decoupling Drake from GraspIt!, Drake will not be linked to GraspIt! and thus not be subjected to the GPL (I am assuming the term "linked" here refers to object linking by the linker during the build process). No?

jwnimmer-tri commented 8 years ago

"Use a program" there means via its command-line interface. Middleware is more complicated.

sherm1 commented 8 years ago

It also says:

Where's the line between two separate programs, and one program with two parts? This is a legal question, which ultimately judges will decide.

That's a reminder that these questions don't actually have answers. Until there is a lawsuit that goes to trial these legal questions remain unresolved. Useful questions here would be "what is the risk to us?" and "what is the benefit?", as well as "how can we reduce the risk?".

hongkai-dai commented 8 years ago

After reading the GraspIt! papers, I think we should be able to write an alternative grasp planner. I quickly summarize the features of GraspIt! as below

Some of the features are on our radars anyway like the collision detection and dynamic simulation, that we plan to add into Drake. The grasp quality function is not hard to compute. I think I can get a sample + optimization based grasp planner working, using our optimization hierarchy.

The risk of working on a planner ourselves is that we spend a lot of time, and eventually find that an optimal grasp coming from the planner is not robust, and we cannot use it in the real application (which I highly suspect is going to happen). As the Amazon Picking Challenge team already mentioned, they did not use the grasp planning.

What are people's opinion on this? Do you think writing a grasp planner ourselves aligns with our long-term goal in manipulation?

liangfok commented 8 years ago

The risk of working on a planner ourselves is that we spend a lot of time, and eventually find that an optimal grasp coming from the planner is not robust, and we cannot use it in the real application (which I highly suspect is going to happen).

For reference, how robust is the grasp planner in GraspIt!?

What are people's opinion on this? Do you think writing a grasp planner ourselves aligns with our long-term goal in manipulation?

:+1:

liangfok commented 8 years ago

Thanks for the explanation. Would you say that we should prioritize developing / acquiring a "skin" that can provide high fidelity and low latency contact sensory feedback of the grasp quality? Without that kind of feedback, I'm afraid any grasp planning will not be robust.

Possible research directions:

  1. GelSight: http://www.gelsight.com/
  2. Tactile sensors: http://wiki.icub.org/wiki/Tactile_sensors_(aka_Skin)
  3. TakkTile sensors: http://www.labs.righthandrobotics.com/takktile-sensors
liangfok commented 8 years ago

OK. +1 for implementing an optimization-based grasp planner in-house and then using it for evaluating / motivating various tactile feedback technologies.

RussTedrake commented 8 years ago

+@ggould-tri and +@Lucy-tri — this is relevant to our discussion the other day.

Lucy-tri commented 8 years ago

+@sammy-tri This discusses GraspIt!

hongkai-dai commented 5 years ago

I don't think we will add graspit or any grasp planner into Drake, as we don't have any immediate usage of these planners. I will close this issue.