Open SeanCurtis-TRI opened 11 months ago
For reference, our meshcat server can serve files now just fine, and indeed already serves the environment map over a http://
URL.
Maybe this is actually finished, except for a few API docs that mention HDR?
The problem
As of #19837,
Meshcat
can specify an environment map. However, the images that are supported (png, jpg) can only support low dynamic range images. This has an appreciable impact on the illumination of the scene and underutilizes the power of environment maps in PBR.Ideally, we should also be supporting typical HDR image formats (rgbe, hdr). These are images that encode not a normalized color space value at each pixel but a radiance value (an amount of light) at each pixel allowing for highly dynamic lighting effects.
meshcat uses three.js as the underlying graphics engine. three.js has an RGBELoader. However, it has been architected differently from three.js's
TextureLoader
which we use to load png/jpg images. TheRGBELoader
(via theDataTextureLoader
) does not appear to accept urls with thedata:/
prefix.This means we can't simply marshall the base64-encoded bits of an image and stream them over to
meshcat
. We need to provide a server that will actually host the image.I've submitted a question on the three.js form to confirm.
The solution
Drake's
Meshcat
class needs to serve as a host more generally for textures, meshes, etc. In addition to unblocking HDR environment maps, this will also allow browsers to enable their caches more intelligently.