Improving RenderEngineGltfClient fidelity by consuming glTF files

[SeanCurtis-TRI]

The problem

Drake currently supports an incredibly impoverished material specification for the OBJ meshes it consumes. Essentially, it only uses a phong illumination model and, even then, only allows specification of diffuse color (or diffuse texture).

The RenderEngineGltfClient implementation is intended to be able to farm Drake state to an external rendering process to create a higher-fidelity rendering of the simulation state. Frequently, model specifications begin with high-fidelity materials (multiple textured parameters or even different rendering models like PBR). Passing them through Drake (and the RenderEngine and SceneGraph APIs) inevitably filters those specifications down to a fisher-price version of the model. The resulting external rendering suffers for this fidelity reduction.

The solution

Given RenderEngineGltfClient broadcasts the Drake scene as a glTF file, it would be nice to support input models as glTF files. If a model is specified as a glTF file, we'd like the full fidelity of that model to be available to the rendering server. The high level proposal (and impact on APIs) would consist of the following:

1. Allow users to specify "mesh" for perception as a gltf file (a change from our current state where it is obj only).
2. Promise that where such meshes are specified, they are included in RenderEngineGltfClient's output in a lossless manner.

Implementation details

Specification

There are a lot of subtleties inherent in this. How do we simultaneously allow URDF/SDF specification that can nevertheless be meaningfully read by other parsers? The answer to this question is still TBD. For now we'll do the following:

Simply specify the .gltf file in the <visual><mesh>... tag.

This will cause all of our visualizers and most of our RenderEngine implementations to yell ("What is a glTF? I'm ignoring it!") However, the RenderEngineGltfClient will happily use it.

In the short-term, it allows for the specification of a high-fidelity representation for external rendering, but it means there will be no geometry with a usable illustration role for that geometry. For now, we recommend the following:

1. Add a second <visual> tag referencing an OBJ version of the gltf file.
2. Add the <drake:accepting_renderer> tag to that visual and provide a nonsense name -- the name of a render engine that definitely does not exist.

The .gltf file will implicitly serve as a perception-only geometry (as only one RenderEngine implementation will use it). The obj file will implicitly serve as an illustration only (as no RenderEngine implementation will have that name). In the future, the second <visual> will become unnecessary as we offer support for .gltf across all visualizers.

Passing input glTF to render server

We will not change the current RenderEngineGltfClient communication protocol. A single glTF file gets broadcast now and that will continue to be the case. What we'll do is guarantee that all of the data in the input .gltf files will be embedded in that broadcast file. (With two exceptions: no skinning and no animation data.)

[SeanCurtis-TRI]

This is a good venue to discuss the future design for how we want to support specification in URDF/SDF/MJCF files.

cc @jwnimmer-tri @zachfang

[jwnimmer-tri]

In terms of input syntax, I think probably #13689 already provides for a lot of good discussion and we could plausibly host the syntax discussions there instead of here.