Unsteady Grasp

[wangjunyi9999]

Hi,

I'm a student and I'm trying to use MuJoCo for picking object with UR5

I'm looking for some help with grasping. When I test the pick task in MuJoCo simulator, it seems good and everything is fine. However, it is unsteady when I try it in my python code. Either the grasp went through the object or the object slipped out when gripper touches it. Interestingly, these cases occur in the same model file.

Here is a model which explains my question:

minimal XML

```XML ```

this is my gripper part code:

minimal XML

```XML ```

Actually, I googled similar issues before, and the solutions are normally like:

1. change the timestep to 1e-4 1e-5
2. add solref=".01 1" solimp=".99 .99 .01" in
3. make the object larger
4. more stable control
5. Emmmm, waiting for your answer now...

Here is a screenshot / video, illustrating my question: the video shows how the gripper went through the object: https://user-images.githubusercontent.com/87897172/227270924-8b8cfbd3-9af6-480e-9eb8-96cbb5b2079a.mp4 the video shows another abnormal issue that the object slipped out from the gripper

https://user-images.githubusercontent.com/87897172/227272755-46eed6ab-0441-484d-aa48-eba6cd25af0e.mp4

the picture shows the object can be held in the gripper very stable (I just change the height of target) . And you can find the force can be shown in the image:

Hope someone can provide good suggestions, I really really have no idea now, because it seems existing collision between the gripper and target.

Thanks in advance!

[wangjunyi9999]

698 I tried all solutions according to this issue.

[cidxb]

I have also encountered this problem, my solutions are:

1. Make sure you adjust the friction coefficient of your grasper. It can be done through the geom type attributes
2. Add <option noslip_iterations="5"> </option> to your XML file head, you can try to modify the number, 5 is fine by me.
3. Add <option><flag multiccd="enable"/> </option> You can try it out. But I am not sure it will 100% help you, because I have also used this UR5 model, and it act strange when I added these options, I think it is linked to how your geom settings.

[wangjunyi9999]

Thanks for your answer. @cidxb However, they do not work for me. My friction values are set '.8 .8 .8' and according to @yuvaltassa suggestions, noslip_iterations cannot be set more than 3. (even if I try to set 5, it does not work ...) I suppose the reasons why simulation is unstable are either model files set or controlling methods. For the second possible reason, maybe my gripper force is too large, so the object is 'pop out'. But when I decrease its gains and low its grasping velocity, it will go through the object, i.e. almost no collision.

[kevinzakka]

Hi @wangjunyi9999, I would suggest you use the Robotiq 2F85 gripper we have in Menagerie here. The collision geoms for the pads are implemented using multiple boxes to generate more contact points. Would be great if you gave it a try to see if it solves your issues.

[zichunxx]

Hi! @wangjunyi9999 Have you solved it? I met the same error but with Franka arm. Would you mind sharing your solution? Thanks!

[peterdavidfagan]

Hi @STAY-Melody,

I am also working on the same item with the Franka arm. I am starting to use the following package to debug.

It looks like the current robotiq model may require some tweaks (here).

I am trying to get the gripper working like it is in this benchmark, if I get there I will follow up here. Equally would be interested if you manage to solve the problem before then.

[zichunxx]

Hi @peterdavidfagan I'm glad to share some of my progress with you.

I tried to use the robotiq attached to the end of the Kuka arm for some grabbing tasks. I believe you've seen the issue I raised. Unfortunately, I haven't found a solution to this erratic grasp caused by robotiq.

The same problem happens with the Franka arm with its default gripper, and I have improved this after fine-tuning some parameters in XML. Here are some methods I have tried:

• Simulate with smaller timestep, such as 1e-3
• Set noslip_iterations="3", condim=6, and cone="elliptic"
• Enable the multiccd flag
• As stated here, increasing the impratio to reduce slippage.
• Increasing the friction factors of the object and the gripper pad.

The above adjustments can improve the grasp stability to some extent. But in my opinion, the most important effect could be the tendon type used to equivalently limit the movement of the finger joints. So, I commented out the tendon type and controlled the gripper joints separately via the position actuators. After some tests, I found the position actuator can provide more steady grasp force than tendon. However, I didn't test it with the pose changing drastically when the object has been grasped, as shown in your video.

The above is my personal speculation about the unsteady grasp. Hope this could help and look forward to some of your insights.

[wangjunyi9999]

Hi! @wangjunyi9999 Have you solved it? I met the same error but with Franka arm. Would you mind sharing your solution? Thanks!

Hi, bro, sorry for the late reply. Honestly, I did not find a good solution for the issue, so I changed to another simulator named Isaac Gym. That one provides more stable examples so that you do not have to deep more about modeling work. I recommend you have a try. Hope it helps you.

[peterdavidfagan]

Thanks @STAY-Melody for the feedback on your iterations towards solving this problem. I'll also share my learnings here shortly and a solution if I get one. In my current implementation it seems that the arm controller is causing issues rather than the gripper, this being said I will be testing grasps once I fix my arm controller.

[zichunxx]

@wangjunyi9999 I'll try Isaac Gym if I still struggle with this problem. I'm sure this issue will be resolved soon with the attention of the developer team. Thanks for your advice.

[yuvaltassa]

Hi all, we took a look at the Menagerie Robotiq gripper. Two issues that we discovered and fixed are:

1. Using a high impratio requires elliptic friction cones, fixed here.
2. The Robotiq has two box geoms per finger which are what is actually colliding. Some of the previously reported issues involved box-box collisions and it turned out that code has some bugs which should be fixed after 7942fe957ea046750f936ac002d49425e31d9dab. To get this fix before the next release you'll need to build from source.

So the Robotiq gripper from the Menagerie should be pretty good now.

@kevinzakka do we actually have the gripper in the OP?

I suspect it needs some work giving it a better collision layer but TBH I don't recognize it.

Closing for now, please reopen if the above is not sufficient.

UPDATE: there is still some remaining issue with the Robotiq, which manifests when the entire gripper is moved quickly, rather than by the intrinsic grip actuators. Tracking this issue here.

[peterdavidfagan]

I'll also share my learnings here shortly and a solution if I get one.

So I ported the following osc controller to my codebase and I am getting better results (I still need to perform source build for the updates but plan to do this soon).

Screencast from 12-13-2023 05:17:29 PM.webm

This being said despite setting gap=margin for geoms it looks like there is a gap between my gripper fingers and geoms (guessing this is due to the gripper model).

I also have had issues with commanding open/close of the gripper where continuously commanding close command can result in subsequent open commands not being reactive, you'll see at the end of the video the gripper fails to open. I'll post what I learn about this tomorrow as I plan to debug, it might be something in my implementation.

Other than this I am working on making the controller as robust as possible, it works well right now for my purposes but I plan to spend more time with it. @STAY-Melody have you tried more complex grasps using the updates to the robotiq model?

Update: lack of reactivity in gripper open/close was due to a bug in my code.

[kevinzakka]

@peterdavidfagan Your gripper physics still looks a bit weird and the collisions seem off. Here's my controller running on my simple XML which combines the Menagerie and Robotiq models together with no modifications:

https://github.com/google-deepmind/mujoco/assets/10518920/83ebd84e-2f45-418a-ac9e-a570a03befd8