meshcat-dev / meshcat

Remotely-controllable 3D viewer, built on top of three.js
MIT License
256 stars 48 forks source link

Add billboard capability #145

Closed kingjin94 closed 8 months ago

kingjin94 commented 1 year ago

Add sprite drawing as a way to add billboards (2D text annotations that always face the camera). Allows to write text unto the scene.

An example:

Screenshot from 2023-07-03 14-26-30

Send json:

{
    u"metadata": {
        u"version": 4.5,
        u"type": u"_billboard",
    },
    u"geometries": [],
    u"materials": [],
    u"object": {
        u"uuid": <some_uuid>,
        u"type": u"_billboard",
        u"text": "test444"
        u"base_width": 75,
        u"size": 16,
        u"global_scale": 0.01,
        u"matrix": np.eye(4).flatten().tolist(),
        u"text_color": "white",
        u"background_color": "blue",
    }
}
jwnimmer-tri commented 9 months ago

I think we can already do this, without any new javascript. Here's a sample message:

        viewer.handle_command({
            type: "set_object",
            path: "/meshcat/sprite",
            object: {
                metadata: {
                    version: 4.5,
                    type: "Object"
                },
                textures: [
                    {
                        uuid: "0c8c99a8-bba8-11ee-b7a2-4b79088b524d",
                        type: "_text",
                        text: "Hello, world!",
                        font_size: 300,
                        font_face: "sans-serif"
                    }
                ],
                materials: [
                    {
                        uuid: "6fe7011b-bba7-11ee-b7a2-4b79088b524d",
                        type: "SpriteMaterial",
                        transparent: true,
                        map: "0c8c99a8-bba8-11ee-b7a2-4b79088b524d"
                    }
                ],
                object: {
                    uuid: "6fe7011c-bba7-11ee-b7a2-4b79088b524d",
                    type: "Sprite",
                    material: "6fe7011b-bba7-11ee-b7a2-4b79088b524d"
                }
            }
        });

The test file https://github.com/meshcat-dev/meshcat/blob/master/test/texture_text.html is along the same lines, but with the text on a shape (plane) instead of a sprite.

jwnimmer-tri commented 9 months ago

In https://github.com/meshcat-dev/meshcat/pull/173, I have a demo that shows this feature might already be available. If you have a chance, please let us know if there's still something missing.

jwnimmer-tri commented 8 months ago

For now, I'll assume that https://github.com/meshcat-dev/meshcat/pull/173 was sufficient. If not, please let us know -- e.g., open a new pull request with more details.