meshcat-dev / meshcat

Remotely-controllable 3D viewer, built on top of three.js
MIT License
250 stars 44 forks source link

Displaying point cloud using a c++ function #81

Open ad1t7a opened 3 years ago

ad1t7a commented 3 years ago

I was writing a c++ wrapper for meshcat and and was successfully able to display meshes. I ran into trouble while displaying a 3d point cloud and have been stuck at it for the past few days. Although it appears that meshcat is able to receive the json object when sent using ZMQ, it isn't displaying anything on the display. My guess is that json format is incorrect. Could you please let me know if I am missing a parameter to display a 3d point cloud (or messing up the expected data format) :

void PointCloud::loadPointCloud(const char *path) {
    double worldPos[3] = {0, 0, 0};
    int colorRGB = 0xFFFFFF;
    std::string geomUID = generateUUID();
    std::string materialUID = generateUUID();
    std::string objectUID = generateUUID();
    // these are the points
    std::array<std::array<float, 3>, 3> arr = {{{0.5207591 , 0.07896397, 0.89273536}, {0.88037544, 0.75724393, 0.33980307}, {0.36066929, 0.9843244 , 0.67521834}}};
    std::stringstream ss;
    msgpack::pack(ss, arr);

nlohmann::json setPointCloudCmd = {
    {"type", "set_object"},
    {"path", path},
    {"object",
    {{"metadata", {{"type", "Object"}, {"version", 4.5}}},
        {"geometries",
            {
                {
                    {"type", "BufferGeometry"},
                    {"uuid", geomUID},
                    {"data", 
                        {"attributes",  
                            {"position", 
                                {"itemSize", 3},
                                {"type", "Float32Array"},
                                {"normalized", 0},
                                {"array", arr}
                            }
                        }
                    },
                }
            }
        },
        {"materials",
            {
                {
                    {"vertexColors", 2},
                    {"size", 0.1},
                    {"color", colorRGB},
                    {"type", "PointsMaterial"},
                    {"uuid", materialUID}
                }
            }
        },
        {"object",
            {
                {"geometry", geomUID},
                {"material", materialUID},
                {"matrix", {1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, worldPos[0], worldPos[1], worldPos[2], 1.0}},
                {"type", "Points"},
                {"uuid", objectUID}
            },
        }
    }}
};
// this part has been tested on other jsons and is known to work
sendZMQ(setPointCloudCmd, true);
  }
ad1t7a commented 3 years ago

Any pointers on how to debug would help too!

RussTedrake commented 2 years ago

If it's useful, I have a c++ implementation here: https://github.com/RobotLocomotion/drake/blob/db541fa1780e3725e0b822c11fd19caaaea09929/geometry/meshcat.h#L132