Rendered point cloud is at a different pose compared to the mesh location

[prajval10]

Hi @Jmeyer1292,

Thanks for the great open-source depth simulator.

Following the example code, I placed the mesh at the origin and visualised the rendered point cloud. It seems to be at another location and not at the origin. The code snipped:

    std::unique_ptr<gl_depth_sim::Mesh> mesh = gl_depth_sim::loadMesh(mesh_path);
    Eigen::Vector3d camera_pos (1.0, 1.0, 1.0) ;
    Eigen::Vector3d look_at (0,0,0);
    const auto pose = lookat(camera_pos, look_at, Eigen::Vector3d(0,0,1));

    gl_depth_sim::CameraProperties props;
    props.width = 640; // In pixels
    props.height = 480;
    props.fx = 550.0f; // In pixels
    props.fy = 550.0f;
    props.cx = props.width / 2; // In pixels
    props.cy = props.height / 2;
    props.z_near = 0.25f; // In "world units"
    props.z_far = 10.0f;

    // Create the depth camera itself
    gl_depth_sim::SimDepthCamera sim (props);

    // Adds mesh to the renderable scene at the origin
    sim.add(mesh, Eigen::Isometry3d::Identity());

    // Render the image and download the depth data
    gl_depth_sim::DepthImage depth_data = sim.render(camera_pose);

    // Lastly, convert it to a point cloud and do something with it...
    pcl::PointCloud<pcl::PointXYZ> cloud;
    gl_depth_sim::toPointCloudXYZ(props, depth_data, cloud);

    pcl::visualization::PCLVisualizer visu("Alignment");
    visu.setBackgroundColor(0, 0, 0);
    visu.setSize(1280, 1024); // Visualiser window size
    visu.addCoordinateSystem(0.5);
    visu.addCoordinateSystem(0.1, pose.cast<float>(), "camera_frame1");
    visu.addPointCloud(cloud.makeShared());
    visu.spin();

Note: the world origin frame is set larger than the camera frame. The mesh is at the origin, the rendered point cloud does not match with the mesh location.

[prajval10]

Found the problem: The generated pointcloud is in the frame of the camera. It needs to brought into the "world" frame through a transformation pcl::transformPointCloud(cloud, cloud, pose).