How to detect the contact and do the limitation in the code?

[wukongwong]

I'm not sure if I'm understanding this correctly, please correct me. The controller does not limit the position error when no contact occurs, it only limits the error when contact occurs. So how are you guys detecting in your code that the robot is in contact with an object and then enabling the position error limiting?

[jianlanluo]

We don't detect contact explicitly, the reference limiting is set so that it won't block free space movement, it will only enforce it whenever in contact. See section 4.5 in the paper: https://arxiv.org/pdf/2401.16013

[wukongwong]

We don't detect contact explicitly, the reference limiting is set so that it won't block free space movement, it will only enforce it whenever in contact. See section 4.5 in the paper: https://arxiv.org/pdf/2401.16013

The key lies in why reinforcement learning outputs a smaller target position value when in contact. If the reinforcement learning strategy outputs a larger action when contacting an object and this action is helpful in completing the task, then this constraint will not come into play, and the controller will still produce a large force, right?

So, I'm thinking, if reinforcement learning tends to learn to produce a small action when in contact, then even without this constraint in the controller, it wouldn't produce excessive force because excessive force would not be helpful in completing the task.

In this case, is there a significant difference in the effectiveness of the controller proposed in the paper compared to a controller without this restriction? Because the controller proposed in the paper's restriction will only be triggered when reinforcement learning produces a smaller action.

[jianlanluo]

No, we don't clip the values from the RL policy, this is doing reference limiting in real-time layer. you don't want to learn to output smaller actions as that's something hard to enforce, your objects will slide from your gripper, get damaged before you learn sensible policies

[wukongwong]

Thanks for your input.