meshcat-dev / meshcat

Remotely-controllable 3D viewer, built on top of three.js
MIT License
256 stars 48 forks source link

Accept embedded gltf files as _meshfile_objects #144

Closed SeanCurtis-TRI closed 1 year ago

SeanCurtis-TRI commented 1 year ago

This accepts embedded .gltf files as _meshfile_objects. It may also accept distributed gltf files as long as the meshcat server respects meshcat's requests for the missing .bin and image files.

This cannot be part of the ExtensibleObjectLoader (EOL). EOL inhertis from ObjectLoader where as GLTFLoader inherits from Loader. The workflow of ObjectLoader is incompatible with GLTFLoader's workflow. Fortunately, GLTFLoader produces a THREE.Group that we can simply stash in the scene.

To support PBR materials (reflective surfaces which reflect an environment map), we change the "background" gradient to support environment textures. This is largely an innocuous change. However, there is a distinct difference between the new background and the old.

In the old, the gradient was registered in screen space. Whether I was looking up or down, the top color was top of the screen and the bottom at the bottom of the screen.

The new environment map now use the "top" (+z infinitely far away) and bottom (-z infinitely far away) of the world. So, rotating the camera to look up sees the upper pole and looking down shows the lower pole.


This change is Reviewable

SeanCurtis-TRI commented 1 year ago

Here are a couple of screen grabs for what this enables (taken from a hacked version of Drake's meshcat_manual_test.cc).

gltf_pr_images Simple, abstract cube with a non-trivial material.

gltf_pr_images2 The canonical "damaged helmet" as featured on Khronos's site.

jwnimmer-tri commented 1 year ago

\CC @pathammer in case you have any suggestions

RussTedrake commented 1 year ago

Thanks @pathammer ! Very helpful.

@SeanCurtis-TRI -- when I run the new test (thank you for adding it), the rendering looks fine by my browser java console warns

main.min.js:2 THREE.WebGLTextures: Unsupported texture encoding: srgb-linear
H @ main.min.js:2
k @ main.min.js:2
R @ main.min.js:2
Bo @ main.min.js:2
da.upload @ main.min.js:2
(anonymous) @ main.min.js:2
renderBufferDirect @ main.min.js:2
Oe @ main.min.js:2
De @ main.min.js:2
Ie @ main.min.js:2
render @ main.min.js:2
update @ main.min.js:2
fromEquirectangularTexture @ main.min.js:2
get @ main.min.js:2
render @ main.min.js:2
render @ main.min.js:2
render @ main.min.js:2
animate @ main.min.js:2
Viewer @ main.min.js:2
(anonymous) @ meshfile_object_gltf.html:13

Should I be worried?

SeanCurtis-TRI commented 1 year ago

Thanks, @RussTedrake.