search

RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.24k stars 1.25k forks source link

Primitive texture coordinates don't align across perception and illustration #20495

Open SeanCurtis-TRI opened 10 months ago

SeanCurtis-TRI commented 10 months ago

What happened?

We allow texture maps to be applied to primitives. These textures appear in the render engine implementations. Effort was made to have the same set of texture coordinates between RenderEngineGl and RenderEngineVtk (and we get RenderEngineGltfClient for free as it inherits from RenderEngineVtk). For historical reasons, the VTK texture coordinates (where provided) have been the de facto definition of Drake's primitives' texture coordinates.

However, the texture coordinates defined by three.js for the primitives does not match VTKs (as they both make arbitrary decisions). It would be better if Drake presented a unified world whether rendering or displaying in meshcat.

The practical solution is to transform the textures in one domain to match the others. Several thoughts:

  1. A simple transformation may be insufficient. The texture coordinates across the mesh may simply be so different that it is impossible to get it looking the same across all faces. The two primitives most likely to suffer from this are boxes and cylinders.
  2. It's not clear whether we should change VTK to match meshcat or vice versa. This has to be determined.
    • We might argue that meshcat should change to match VTK because VTK has long supported textured primitives and there may be applications where people have come to rely on the VTKs approach. This is a particularly meaningful argument because, at the time this issue was crated, meshcat isn't actually getting the texture maps (#19076). At a minimum, it's a good reason to bias ourselves in this direction.
    • On the other hand, we might find manipulating the RenderEngines implementations to be simpler as they live in the Drake process.

Version

No response

What operating system are you using?

No response

What installation option are you using?

No response

Relevant log output

No response

SeanCurtis-TRI commented 10 months ago

Note: back when the only visualizer was drake_visualizer this wasn't a problem as both visualizers used VTK as their rendering back bone.

jwnimmer-tri commented 10 months ago

For reference -- are the primitive texture coordinate conventions documented (promised) in any of our API materials anywhere?