RobotLocomotion / drake

Model-based design and verification for robotics.
https://drake.mit.edu
Other
3.35k stars 1.27k forks source link

Meshcat support for set_object with object.textures.type=_text #20884

Open jwnimmer-tri opened 9 months ago

jwnimmer-tri commented 9 months ago

Is your feature request related to a problem? Please describe.

Upstream Meshcat.js has the capability to use text as a texture. (See here for a demo.)

However, the Drake C++ Meshcat API does not offer any way to transmit the set_object message with those fields.

Describe the solution you'd like

Add another Meshcat::SetObject() overload to send this message. Something like:

 /** Sets the 3D object at a given `path` in the scene tree to the given shape
  whose material will be the given text (rendered onto canvas).
  The shape must be a primitive shape (not a mesh file). */
  void SetObject(std::string_view path, const Shape& shape, std::string_view text,
                 const Rgba& rgba = Rgba(.9, .9, .9, 1.));

Describe alternatives you've considered

Allow the user to give us arbitrary JSON for the object in a set_object and we could just pass that along.

jwnimmer-tri commented 9 months ago

Note to self: the color property that ends up changing the text color is named emissive:

--- a/test/texture_text.html
+++ b/test/texture_text.html
@@ -46,6 +46,7 @@
                         uuid: "6fe7011b-bba7-11ee-b7a2-4b79088b524d",
                         type: "MeshPhongMaterial",
                         transparent: true,
+                        emissive: 15649962,
                         map: "0c8c99a8-bba8-11ee-b7a2-4b79088b524d",
                         // Make the text visible front and back (double sided).
                         side: 2
jwnimmer-tri commented 9 months ago

I still think maybe we should add the sugar for sending text, but now I'm leaning a bit more toward the second option:

Allow the user to give us arbitrary JSON for the object in a set_object and we could just pass that along.

That would open the door to using anything from https://threejs.org/docs/ -- there are a lot more kinds of geometries and objects and materials defined there than we really want to wrap for ourselves. The general-purpose escape hatch would solve a lot more use cases.

Something like:

 /** Sets the 3D object at a given `path` in the scene tree to the given shape,
  per the https://threejs.org/docs/#api/en/loaders/ObjectLoader threejs format. */
  void SetObjectRaw(std::string_view path, std::string_view threejs_object_json);
SeanCurtis-TRI commented 9 months ago

~Humorously:~

~Here in meshcat_manual_test. we animate the opacity of an object - but that property is referred to as material.opacity.~

~So, we already have this feature for animations because the three.js logic can map the property recursively.~

This comment ended up in the wrong issue. Sorry 'bout that.

jwnimmer-tri commented 9 months ago

I decided to file the "raw JSON" as a separate feature request (#20941).

Once that's done, we can circle back here and decide whether we want either: