Detection of self-collisions

[tsampazk]

Is your feature request related to a problem? Please describe. When working on an RL problem with a robotic arm, self-collisions cause a significant slow down on the training process. Here is the related PR with related comments

Moreover, it would be a natural termination point for a training episode when a self-collision happens which is something that probably needs to be avoided in the physical world, and thus the RL agent needs to be aware of it.

Describe the solution you'd like I can imagine a robot method that when called it will return a boolean value that is true when self-collision is happening in that step.

Describe alternatives you've considered Maybe utilize a physics plugin as described here.

Additional context You can see related screenshots of the slow-down and issues in the PR linked above.

Cheers! 😀

[Simon-Steinmann]

For robotic arms I would highly recommend using collision aware inverse kinematics solvers. They can be very fast and avoid the problem all together. Rather use the RL to send IK commands (endeffector pose).

Check out this video of a RL project I did: https://drive.google.com/file/d/10tYRvi3EHlTxF2VFEGBE4TL6-sexYP58/view?usp=sharing

The agent (SAC + HER) sends velocity actions in cartesian x, y, z. Sparse reward when it reaches the randomized green sphere area. The blue box is the operating limit. I used an IKFast solver for the IK. It only gives valid poses and takes less than a millisecond to compute.

[tsampazk]

Hello @Simon-Steinmann and thank you very much for your suggestion! To be honest, i have minimal experience in using IK solvers, but will look into the solution you suggested as it seems quite promising.

In my opinion it would also be a very interesting problem to tackle via RL while trying to achieve another goal. For example in the cartpole problem here, i noticed that when trained, if you pull the pole to the sides, when the cart reaches the arena edge it tries to avoid it (which is basically a 'side' task other than balancing the pole) while doing its best to balance the pole. This behaviour emerges from the episode termination condition of reaching the arena edge. Similarly, an RL agent controlling a robotic arm could incorporate the self-collision avoidance, making it a more end-to-end solution, which tends to be preferred in the field, i think.

[lukicdarkoo]

Moreover, it would be a natural termination point for a training episode when a self-collision happens which is something that probably needs to be avoided in the physical world, and thus the RL agent needs to be aware of it.

You can detect collisions by doing something like this: https://github.com/cyberbotics/webots/blob/9a71190bb2084fec04246bd1008d9d311bef656f/projects/samples/robotbenchmark/obstacle_avoidance/controllers/supervisor/supervisor.py#L65-L72

Indeed, it would be probably handy to have something like wb_supervisor_node_does_self_collide, but we should think about it.

When working on an RL problem with a robotic arm, self-collisions cause a significant slow down on the training process. Here is the related PR with related comments

This is expected and I don't think there is a way around this. Collision detection is usually done on a single CPU core (since all your objects are close to each other it is not parallelized), so your CPU core has to handle the physics simulation and the collision detection. If you think about it, even a simple box-box collision is not trivial (consider rotations), so if you do it with meshes it is extremely hard to compute. You should definitely simplify it with simple bounding objects. Or, in the worst case, simplify the meshes used as bounding objects.

[tsampazk]

Thank you for your suggestions @lukicdarkoo.

Of course, collisions are a complicated issue and slow-downs are to be expected. The feature suggested is more of a "nice to have" thing and at least for this specific case not critical. Removing the bounding boxes altogether can work fine for the use-case!

[Simon-Steinmann]

Similarly, an RL agent controlling a robotic arm could incorporate the self-collision avoidance, making it a more end-to-end solution, which tends to be preferred in the field, i think

Existing IK solutions already avoid self collision and even environment collision when set up properly. I dont think there is much value to be added in this area. Existing solutions are already very powerful and fast. Actually learning tasks, that is where it is at. Perhaps combining it with computervision etc.

[lukicdarkoo]

@tsampazk Can you try decreasing the softCFM parameter to like 1e-5? You need to create a ContactProperties node in WorldInfo > contactProperties.

[BenjaminDeleze]

I'm closing this for now, feel free to reopen it if you have additional issues.