Epic [geometry] Sync visual and collision geometries for deformables

[DamrongGuoy]

Description

Traditionally we use visual geometry and collision geometry in a rigid body simulation, and when we introduce deformable objects, we will have to apply the same deformation on the collision geometry to the visual geometry as well.

Possible solution

We might have to work on the representation of visual geometry to accommodate deformation. We might need to use TriangleSurfaceMesh instead of shape specification. For example, users may specify a visual geometry as Mesh("fine_visual_mesh.obj") and a collision geometry as Mesh("coarse_fast_contact.vtk"). When the collision geometry makes contact and deforms, we will apply that deformation to the TriangleSurfaceMesh of the visual geometry and maintain its visual property (texture coordinates, visual materials, etc.).

Applying deformation defined on the tetrahedral volume mesh of the collision geometry to the triangle surface mesh of the visual geometry is similar to interpolation/extrapolation of vector fields.

• [ ] In-memory surface mesh representation of visual geometry and associated visual properties like textured coordinates and PBR (physically based rendering) properties.
• [ ] Vector-field interpolation/extrapolation from tetrahedral meshes to triangle meshes.
• [ ] Passing data between related modules (multibody, proximity, illustration, rendering/sensors).
• [ ] Meshcat visualization.

Alternative

Instead of the in-memory representation of visual geometry, we might continue to use out-of-core representation by Wavefront Obj files. In that case, we can maintain a temporary TriangleSurfaceMesh, apply deformation, and write out a new Obj file. Then, we dispatch the load message to Meshcat. It might be a good intermediate solution since changing data structures is never fun.

The other alternative is doing nothing. We can just assume the visual mesh and the collision mesh are the same for now. The only downside is that a green bell pepper and a yellow bell pepper will both look white.

[DamrongGuoy]

CC: @xuchenhan-tri

[jwnimmer-tri]

To me, the natural answer would be to use same Shape with both illustration and proximity geometry properties Mesh("coarse_fast_contact.vtk") and then texture it using illustration properties instead of an MTL file. Do *.vtk files support texture coordinates as an extra per-vertex channel? If not, is there any other reasonable data format for tet meshes with uvs?

Edit: I just read the VTK file format and it does support uvs and normals, at least to some extent.

[DamrongGuoy]

I agree that it is beneficial to assume every vertex of Obj triangle mesh (Illustration) is a vertex in VTK tetrahedral mesh (Proximity). It will simplify the problem. It will let us easily deform the illustration geometry following the proximity geometry.

@jwnimmer-tri I'm not sure what "texture it using illustration properties instead of an MTL file" means. From the class documentation IllustrationProperties, I can't tell what is available for illustration properties.

[jwnimmer-tri]

Things you might say in an MTL file you can often instead say using illustration properties. For example, map_Kd is (phong, diffuse_map):

https://github.com/RobotLocomotion/drake/blob/11ce429e420f022bbc8e15663bbafd74e076f13f/multibody/parsing/detail_sdf_geometry.h#L89-L95

It does mean using a single material for the entire surface, but hopefully that is sufficient.