Gripper teleoperation

[arsh09]

Hello.

I am attempting to do Franka (leader) and Franka (follower) teleoperation using ROS on multiple computers and position mapping.

The arms are working as expected. However, I am having a hard time to make gripper teleoperate. Considering that there are no force sensing on the gripper, I start both gripper in position mode then I do homing on both the grippers i.e. on leader and follower side. Then the operator starts moving the gripper on leader side which gets mimcked on the follower side using Gripper Move action client (for follower) with leader gripper position as set point. This works as expected as well.

However, when the user starts picking an object, the move action hangs completely and the robot stops in error. I understand that the gripper control is not realtime similar to arm control and there may be a while loop running for the gripper in it gripper move action server but I am unable to find a solution as how can I exit the loop before an error occurs.

Any suggestions as how to avoid this move action "hangs" when the operator is trying to pick an object?

Thank you

[gollth]

Hi @arsh09

yes you are correct, the gripper control is not real-time capable. I'm not 100% sure if I understand your problem. You are using franka_ros & ros_control to control the arms correct? Which exact controller are you using, joint position?

In theory the gripper action server & the arm control run on separate nodes (processes) franka_gripper_node & franka_control_node, respectively, so shouldn't block each other.

[gollth]

when the user starts picking an object

Also what exactly do you mean by that? Are you calling /franka_gripper/move in that case?

[gollth]

One more comment regarding your application: Since we don't have force sensing on the gripper, you might want to consider using the GraspAction instead of Move. With move your follower might never reliably apply grasping forces onto the object. Of course that is not entirely teleoperation anymore, since you have to kind of interprete a positional change on the leader fingers as grasping action.

Depends on your use case of course, just my two cents =)

[arsh09]

Hi @gollth

Thank you for your suggestions.

On leader (arm) side, I am using Cartesian Impedance controller with zero null space and cartesian space stiffnesses. One follower (arm) side, I using joint position controller (from ros_control). As I start franka state controllers as well as joint state controllers, the position mapping for leader to follower arm is easy and is working.

For gripper, I am starting both gripper state and position controllers. I understand there are 5 action servers for gripper (Move, Grasp, Homing, Stop and GripperCommand). I tried using GripperCommand and Move (separately, one by one) on follower side to teleoperate. Move action works on follower side i.e. if I do homing on both gripper first, and then the operator starts moving gripper on leader side, the follower gripper mimicks it. However, if the operator tries to pick an object and closes the finger on leader side, the follower side gripper close its finger but then as soon as the finger touches the object (or probably squeeze it a little), the whole thing hangs. I am guessing that it may be that Move action on follower gripper side is trying to go to a width which is smaller than the object's width and its in some kind of loop. I thought that using Stop action on follower side, I could exit/cancel the Move gripper action but that does not happen (all the time).

I will try using grasp action follower side and keep track of leader side gripper position (and velocity). That may solve the issue.

Out of curiosity, is it not possible for get access to motor torque/current of gripper. That may help to provide some kind of signal to user that okay gripper can not close after that. I suppose internally, Grasp action is calculating the applied force similarly?

Thanks

[gollth]

Thanks for the detailed information.

I am guessing that it may be that Move action on follower gripper side is trying to go to a width which is smaller than the object's width and its in some kind of loop.

Yes that is exactly what happens. The Move action is really designed to move the fingers to a predefined position, not to grasp objects.

AFAIK cancelling the action or calling the stop action will exit this loop, though. I tried to reproduce your "hanging" behavior with one robot which has a object in between while grasp is active, but couldn't. I was able to stop the move action both with stop and cancellation. Maybe it's something related with your dual arm setup?

is it not possible for get access to motor torque/current of gripper.

Exactly, this is not possible

[gollth]

One more comment: We have tried a similar demonstration at one point. Maybe have a look at teleop_gripper_node and see if you can take inspriation from that.

[arsh09]

Hi @gollth

Thank you for your suggestion about the teleop_gripper_node. I can not believe that I missed it in franka ros package.

I have just tried it and it works to my requirements. Though its not completely teleoperated, at the moment it is sufficient for grasping objects.

However, I would still be a little curious as why the access to gripper motor torque is limited? that could really help in providing some kind of haptic feedback on leader side (or in other applications).

Thank you

[gollth]

No worries, glad it helped you!

However, I would still be a little curious as why the access to gripper motor torque is limited? that could really help in providing some kind of haptic feedback on leader side (or in other applications).

Yes definitely, many applications would benefit from that! The problem is an architectural one how the gripper is controlled over the internal bus. Since this is propriety, unfortunately I cannot go into too much detail here.