Object not getting attached to Robotiq 2 fingered gripper

[aaky]

Hello @JenniferBuehler I tried using your plugin for my setup of UR5 arm with Robotiq 2 fingered gripper for grasping cube of 50 mm length and breadth with no success on Gazebo 2 since I have ROS Indigo installed. Here is my Plugin config settings.Kindly guide me through the same

ur5 robotiq_85_left_inner_knuckle_link robotiq_85_right_knuckle_link robotiq_85_right_finger_tip_link robotiq_85_left_finger_tip_link 100 10 10 20 0.005 true __default_topic__

[LuserBen]

Hi, @JenniferBuehler . Maybe I find a reason why sometimes object slips out of the hand for robotiq two finger gripper.

If the object is in the middle of the gripper(x = y, because I get the object state by gazebo service /get_model_state, there is no position error of the object), left finger and right finger will arrive target pose at the same time but without enough collision and counting contact points in 'close' process. So I add a position offset (x != y), one finger will contact the object first, then another finger. It works.

As I described in https://github.com/JenniferBuehler/gazebo-pkgs/issues/18. In matlab I get the object position by kinect v1, various errors lead to x != y. In C++ the exact position of the object can be get ( x!=y). So for me why matlab client worked, but C++ client failed.

I am not sure if it will affect.

[JenniferBuehler]

Hi,

thanks for posting this. I'm not sure I quite 100% follow what you mean. So you are saying that with your robot, if the grippers both touch the object at the same time, then it fails, but if one touches it first and then the other, then it succeeds?

Cheers Jenny

[LuserBen]

Hi, @JenniferBuehler . Yes. If the gripper both touch the object at the same time, object will always slip out (not every time) without a collision process (By the way, for a 6cm×6cm×cm cube I set the robotiq joint value to 0.266, two fingers will touch the object while they are stopped at the same time). If one touches object first and then the other, there is a violent collision process in gazebo GUI and Terminal will print “GazeboGraspFix: Attach cube ······".

So I guess it have an effect on grasping performance.

Cheers, Lu

[JenniferBuehler]

Hi Lu,

ok that's very strange indeed. I'll re-open this so I can try to reproduce it and see what the reason for this may be. It will be a while yet before I can look into this though because I have to first finish other work. If the repository you are using to test this is public, could you please send me a link, then it will be easier for me to verify that.

Cheers Jenny

[LuserBen]

Hi, @JenniferBuehler . Haha, I'm not sure if it is common for others, I just provide a solution for people who meet the same problem. So it is not an urgent matter. I have not uploaded my repository yet, maybe after a few months, but I find a repository is similar to me, https://github.com/neka-nat/ur_ws, you can replace the object with a cube. Thanks for your consideration.

Cheers, Lu

[JenniferBuehler]

Hi Lu,

sounds good, thank you! I will look into this as soon as I can. Thanks for the link.

Cheers Jenny

[caiobarrosv]

Hello, @LuserBen.

I have the same exact problem. I get the final position and orientation of the object to be grasped by a CNN and it is not very precise. As a result, one finger will touch the object first and the object slips. How did you manage to solve this issue?

[LuserBen]

Hello， @caiobarrosv, I just added an offset (for example, x+3mm) to the position of object so that two fingers won't touch the object at the same time. It works for me.

[rhowsane]

Hi! @JenniferBuehler

I’m facing difficulty to attach my object using your pluggin. Could you help me please? Here are the proceedings that I adopted to investigate the problem:

1. Copy the files gazebo_grasp_pluggin, gazebo_version_helpers and general-message-pkgs to my src directory. After catkin_make

2. I inserted the parameters at panda_arm_hand.urdf.xacro to point to the gzplugin_grasp_fix.urdf.xacro

3. I had tested if the function UPDATE was being published uncommenting some messages and after doing the command ‘catkin_make’ at the repo. It was working well.

4. I tested equal to 0 (to verify if the detachement was happening quickly), to 0.005 (the initial configuration that wasn’t working) and to 0.01. The last one to garantee that even if the rightfinger could move away from the mobile 1cm, I would still have a contact. But what happened in the end was that the robot would spend more time attaching the mobile and after It would dettache it as we can see at the video.

https://gfycat.com/heftydefiantcleanerwrasse

I launched the simulation and then I got the first screenshoot. When I was verifying what was happening I took the second one. From which I realized that the leftfinger was the first to touch the mobile and also the last to lose contact. The X spawn position of the mobile was 0.50 m and the goal position for the grasping the same.

The main question in my head is why the gripper consistently loses the contact after a certain time.

If you have any ideas of what I could do to fix my grasping, It would help me a lot.

[JenniferBuehler]

Hi @rhowsane,

Hm that sounds strange that it would take longer to attach with a higher release tolerance... Can you try a hyper exaggerated value like 1 to confirm this behaviour?

It's a bit hard to see in the video whether it attached at all, but the extract from the output suggests so, though I can't see it all... Could you maybe post the output as text file instead of image?

If you're already going into the code enabling debug prints, that's probably the best bet to follow the lead on what's going wrong. There's a number of commented debug prints which could be helpful

[Harshal82More]

Hi @JenniferBuehler, I hope you are doing fine. I would need your help regarding some issues while grasing. I have created my own robot manipulator. I used your plugin seems to be loaded in msg. I have an object (payload) to lift but it slips from fingers. I have few questions - 1. when I use position controller it slips from fingers. I do not get any output while gz topic -e ~/gazebo/default/mrm/contacts.. 2. Some references told me to use effort controller. Some how effort controller doesn't work for fingers will. May be I need some tuning. I am getting sophisticated way to do it. Your suggestions will be helpful.

Thanks

[JenniferBuehler]

Hi :) you mean that even when the robot is touching the object, you don't get output for the contacts on the topic? Did you set up collision shapes for fingers and object?

[Harshal82More]

I have 4 fingers at EE At first I started with positioncontroller/trajectory controller. some how objects slips from finger. Your plugin seems to be in output even I have rostopic list robot/contacts. Even finger is touching object I dont get any output. When I change to effort controller/tunning for manually getting difficult.

Can you tell me which is correct approach position or effort?

[JenniferBuehler]

If you don't get output for the contacts (via gz topic) it could mean you have no collision shapes (eg only visual). It then doesn't matter if the plugin is loaded or not. Did you add them in the urdf?

[Harshal82More]

Hi Jenifer, thanks for writing. Yes I have collision already added. I just modified my world file and I have gazebo contacts topics updating. It seems to be working fine. But when I deploy object on ground it seems the contact between ground and objects displaces platform with arm. Any suggestions what I can work one to tune this? Reducing effort or something? thanks Harshal

[JenniferBuehler]

Hi Harshal, Bit difficult to estimate what's happening from what you're saying, if I understand correctly, you place an object on the ground next to the robot, and then the robot moves? Try to place the object in a position in which it doesn't collide already with the ground maybe (eg 1mm above ground so that it falls on it), or make it part of the world, or make it static (if it's not meant to be moved again). Hope something like this resolves it Jennifer

[Harshal82More]

Thanks @JenniferBuehler this trick worked. That was the same issue what I you understood. I drop it few 2 mm above ground.

[JenniferBuehler]

Glad it worked :)

[BenRJG]

@JenniferBuehler Hi! Sorry to be adding to an old issue (I can open a new one if it'd be preferable), but I'm having the same issue as the item not gripping and no outputs :'D Here's a short video showing it in action. I've tried to follow this post from the top but I struggle to find anywhere to change the topic type, and the line to un-comment for terminal output has changed. Any help would be really appreciated :)

Log on load. I've also checked gazebo collision output and values are showing on collide

[Msg] Loading grasp-fix plugin
[Msg] GazeboGraspFix: Using disable_collisions_on_attach 1
[Msg] GazeboGraspFix: Using update rate 10
[Msg] GazeboGraspFix: Using max_grip_count 20
[Msg] GazeboGraspFix: Using grip_count_threshold 10
[Msg] GazeboGraspFix: Using release_tolerance 0.005
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_left_gripper_left_follower::tracebot_left_gripper_left_follower_collision
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_left_gripper_left_follower::tracebot_left_gripper_left_follower_fixed_joint_lump__tracebot_left_gripper_left_pad_collision_1
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_left_gripper_right_follower::tracebot_left_gripper_right_follower_collision
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_left_gripper_right_follower::tracebot_left_gripper_right_follower_fixed_joint_lump__tracebot_left_gripper_right_pad_collision_1
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_right_gripper_left_follower::tracebot_right_gripper_left_follower_collision
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_right_gripper_left_follower::tracebot_right_gripper_left_follower_fixed_joint_lump__tracebot_right_gripper_left_pad_collision_1
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_right_gripper_right_follower::tracebot_right_gripper_right_follower_collision
[Msg] GazeboGraspFix: Adding collision scoped name robot::tracebot_right_gripper_right_follower::tracebot_right_gripper_right_follower_fixed_joint_lump__tracebot_right_gripper_right_pad_collision_1
[Msg] Subscribing contact manager to topic ~/robot/contacts
[INFO] [1637314470.329002, 0.000000]: Controller Spawner: Loaded controllers: joint_group_pos_controller
[Msg] Advertising grasping events on topic grasp_events
[INFO] [1637314470.360639, 0.000000]: Loading controller: pos_joint_traj_controller
[spawn_gazebo_model-4] process has finished cleanly
log file: /home/astech/.ros/log/023af28e-4910-11ec-8d07-bf8946c88163/spawn_gazebo_model-4*.log

Plugin config

  <!-- Gazebo Gripper -->
  <gazebo>
    <plugin name="gazebo_grasp_fix" filename="libgazebo_grasp_fix.so">
      <arm>
        <arm_name>tracebot_left_arm</arm_name>
        <palm_link> tracebot_left_gripper_left_spring_link </palm_link>
        <palm_link> tracebot_left_gripper_right_spring_link </palm_link>
        <gripper_link> tracebot_left_gripper_left_follower </gripper_link>
        <gripper_link> tracebot_left_gripper_right_follower </gripper_link>

        <!--palm_link> tracebot_left_arm_wrist_3_link  </palm_link-->
        <!--gripper_link> tracebot_left_gripper_left_driver </gripper_link-->
        <!--gripper_link> tracebot_left_gripper_right_driver </gripper_link-->
      </arm>
      <arm>
        <arm_name>tracebot_right_arm</arm_name>
        <palm_link> tracebot_right_gripper_left_spring_link </palm_link>
        <palm_link> tracebot_right_gripper_right_spring_link </palm_link>
        <gripper_link> tracebot_right_gripper_left_follower </gripper_link>
        <gripper_link> tracebot_right_gripper_right_follower </gripper_link>

        <!--palm_link> tracebot_right_arm_wrist_3_link  </palm_link-->
        <!--gripper_link> tracebot_right_gripper_left_driver </gripper_link-->
        <!--gripper_link> tracebot_right_gripper_right_driver </gripper_link-->
      </arm>
      <forces_angle_tolerance>100</forces_angle_tolerance>
      <update_rate>10</update_rate>
      <grip_count_threshold>10</grip_count_threshold>
      <max_grip_count>20</max_grip_count>
      <release_tolerance>0.005</release_tolerance>
      <disable_collisions_on_attach>true</disable_collisions_on_attach>
      <contact_topic>__default_topic__</contact_topic>
    </plugin>
  </gazebo>

[JenniferBuehler]

Hi @BenRJG, I'll try to find some time on Monday to look at it in more detail (unfortunately won't get time before). Spontaneously the only thing that comes to mind is that collisions should definitely be occurring (from the video). Maybe it's not lasting long enough / stable enough for the plugin to catch.. though I doubt it but worth a try: Did you try to decrease the grip count threshold?

Can you send the output showing on collision that you meant?

Cheers Jennifer

[BenRJG]

Hi @JenniferBuehler Thanks for getting back to me :) Looking slightly better now, I decreased the count threshold to 1 and also tried it with a rectangular shaped object to help with stability and it is attaching, just now its attaching/detaching then the object slips out

[Msg] GazeboGraspFix: Attaching unit_box::link::collision to gripper tracebot_left_arm.
[Msg] GazeboGraspFix: Detaching unit_box::link::collision from gripper tracebot_left_arm.
[Msg] GazeboGraspFix: Attaching unit_box::link::collision to gripper tracebot_left_arm.
[Msg] GazeboGraspFix: Detaching unit_box::link::collision from gripper tracebot_left_arm.

[BenRJG]

Looking at it in action, it'll grip, disable collision (I've set this in the config of your plugin), then begin moving into the object, then it'll detach, cause collision, and repeat until the gripper being in the objected forces it to move away

[BenRJG]

I've tried to increase the release tolerance which has fixed the issue so now it stays gripped, however its now not releasing when the gripper is open! I'm going to play with this a little to see if theres a sweet spot. Please let me know if you have any other suggestions :)

[BenRJG]

Managed to get it to drop the box now too by increasing the count threshold again

      <forces_angle_tolerance>100</forces_angle_tolerance>
      <update_rate>4</update_rate>
      <grip_count_threshold>2</grip_count_threshold>
      <max_grip_count>8</max_grip_count>
      <release_tolerance>0.015</release_tolerance>
      <disable_collisions_on_attach>true</disable_collisions_on_attach>
      <contact_topic>__default_topic__</contact_topic>

Going back to the cylindrical object it's still not liking this very much and refusing to grip :/

EDIT: (Didn't want to keep spamming replys!) The object did end up gripping but took some time to do so (at which point it'd already glitched into an odd position)

[JenniferBuehler]

Hi Ben,

Thanks for the details, I think that's helped me to understand better what the situation is.

What was it originally doing when you had disable_collisions_on_attach to false? If you have this on true, it is particularly important that the grippers stop moving as soon as the object is gripped (otherwise the grippers move too far and a release is triggered, which is what's happening).

In the real robot, ideally some force-feedback would tell it to stop applying pressure on the object. But in practice often each grasp is tailored for the particular object, so the grippers just won't close more than needed to grasp the object (so that it's not squashed). If you stop moving the grippers at a position where the object just about fits between the grippers, the plugin should also not release the object again, and the object should remain attached (and you can probably leave disable_collisions_on_attach on false).

Both (a) moving the grippers into the object (by disabling collisions and keep the grippers closing) and (b) applying too much force on the object (enabled collisions while still moving grippers --> usually leads to wobbling or even exploding robots) are not right. So either of (a) or (b) should be avoided in the first place.

You can try to either

• react to the grasp by subscribing to the attached-state message (see e.g. the ROS code here) and then stop moving the grippers as soon as the object is attached. Or you can
• never close the grippers much more than what is required for the contacts to happen with this particular object (that's how it's usually done).

If you already are considering this, let me know. If not, I would try this first, with the default parameters (e.g. a grip count of 1 is not very stable either, it means it will instantly attach once the gripper touches, but usually what you want is for the condition to be true for at least a few iterations, then a good/stable position is ensured). And only if that doesn't help, then try to fine-tune by adjusting parameters afterwards.

[BenRJG]

Hi Jenifer, Thanks for all the useful info! I have managed to get it to work now, the issues seemed to be the collision of the object was a detailed mesh so I changed this to just a cylinder which helped stabilize when being gripped and preventing the strange collision issues. I agree that the gripper should stop when grasping the object, I'm not actually working on the grasp control for this project so my main aim was just to get something working to demo the functionality in the virtual environment, so I imagine this will work much better when the proper grasp control is implemented! I am going to try this again with the disable collisions back to false as the initial issues again was with the mesh of the object being picked up.

[JenniferBuehler]

Hi Ben,

That's good news! I didn't even think about the possibility that it could be a mesh ;) Yes collisions with meshes are a beast. Let me know how it goes! Jennifer

[yding25]

I have the same issue: the 'OBJ CONTACT' cannot be detected. After a few days of debugging, I find the reason is ROSPY.SLEEP() IS NOT ENOUGH!!! Please, ensure enough processing time for the contact detection. I hope this answer can save you time!

[mco528]

https://github.com/JenniferBuehler/gazebo-pkgs/assets/132716999/f2aede17-79eb-4b24-96a2-f9f2fbb64477

Hi Jennifer. I'm trying to make 3 DOF robots that are voice controlled. I'm trying to get the robot to pick and place, but as seen in the video, the object doesn't get between the grippers. It looks as if the grippers are stuffed. how can i solve this problem

[JenniferBuehler]

Please make a new issue for this, I will respond as soon as I get time to take a proper look