Fail to create a GelSight around a cylinder

[jc-bao]

Hi, everyone. I am trying to create a GelSight in a cylinder. I try to write a sensor configuration file according to the OmniContact example. But the result turns out to be a blank image. Here is the link to my sensor configuration file.

A simple demonstration to the sensor	Gelsight image

What could lead to that error? Any information will be highly appreciated!

[wx405557858]

Hi @jc-bao, Yeah, it may need some tuning for setting up new sensors. The orientation can be tricky.

For debugging, it is highly recommended to temporarily replace the offscreen rendering to the on-screen viewer to see the whole sensor mesh and camera view in PyRender with an interactive window. This helps adjusts the setting to the desired poses.

In TACTO, the offscreen rendering is around here: https://github.com/facebookresearch/tacto/blob/a21d0c4626d74a546d94859226c2fea348babb6a/tacto/renderer.py#L568

For on-screen viewer, the PyRender document is here: https://pyrender.readthedocs.io/en/latest/api/index.html#on-screen-viewer

[jc-bao]

Thanks for that advice! It seems that this is caused by a solid mesh file. After skeletonizing the model, rendering works fine. But when the object touches the Gelsight, the gel sight behaves weirdly. There is a blue light blink.

[jc-bao]

I think I have solved this problem by defining the different mesh in the sensor and robot models. In the robot URDF model, I use the solid model. For the mesh in sensor configuration, I use the hollow model, which only contains the surface. The previous problem might be caused by the small thickness of the object's surface.