TSDF volume integration error

[Alihamdy2496]

TSDF volume integration error I am working on a 3d reconstruction project and built the structure from motion part, I get relatively good results when try to use TSDF volume integration to get dense reconstruction i get this error

RuntimeError: [Open3D Error] (void __cdecl open3d::pipelines::integration::ScalableTSDFVolume::Integrate(const class 
open3d::geometry::RGBDImage &,const class open3d::camera::PinholeCameraIntrinsic &,const class 
Eigen::Matrix<double,4,4,0,4,4> &)) D:\a\Open3D\Open3D\cpp\open3d\pipelines\integration\ScalableTSDFVolume.cpp:78: 
[ScalableTSDFVolume::Integrate] depth image size is (640 x 480), but got (-1 x -1) from intrinsic.

here is my calculated camera poses views.txt and this is a google drive link link containing RGB images and depth images and here is my code

import open3d as o3d
import numpy as np
import pyvista as pv

class CameraPose:
    def __init__(self, mat):
        self.pose = mat

    def __str__(self):
        return "Pose : " + "\n" + np.array_str(self.pose)

def read_trajectory(filename):
    traj = []
    with open(filename, "r") as f:
        metastr = f.readline()
        while metastr:
            mat = np.zeros(shape=(4, 4))
            for i in range(4):
                matstr = f.readline()
                mat[i, :] = np.fromstring(matstr, dtype=float, sep=" \t")
            traj.append(CameraPose(mat))
            metastr = f.readline()
    return traj

camera_poses = read_trajectory("views.txt")

volume = o3d.pipelines.integration.ScalableTSDFVolume(
    voxel_length=4.0 / 512.0,
    sdf_trunc=0.04,
    color_type=o3d.pipelines.integration.TSDFVolumeColorType.RGB8,
)
folder = "RGB_depth"

for i in range(len(camera_poses)):
    print("Integrate {:d}-th image into the volume.".format(i))
    color = o3d.io.read_image("{}/frame-%06d.color.jpg".format(folder) % (i))
    depth = o3d.io.read_image("{}/frame-%06d.depth.png".format(folder) % (i))

    rgbd = o3d.geometry.RGBDImage.create_from_color_and_depth(
        color, depth, depth_trunc=4.0, convert_rgb_to_intensity=False
    )
    intrinsics = o3d.camera.PinholeCameraIntrinsic()

    intrinsics.intrinsic_matrix = [
        [1449.9962, 0.0, 974.9934],
        [0.0, 1441.6412, 453.0694],
        [0.0, 0.0, 1.0],
    ]

    volume.integrate(
        rgbd,
        intrinsics,
        camera_poses[i].pose,
    )

print("Extract a triangle mesh from the volume and visualize it.")
mesh = volume.extract_triangle_mesh()
mesh.compute_vertex_normals()

v = np.asarray(mesh.vertices)
f = np.array(mesh.triangles)
f = np.c_[np.full(len(f), 3), f]
mesh = pv.PolyData(v, f)
plotter = pv.Plotter()

plotter.add_mesh(
    mesh,
    show_edges=True,
    color=[0, 1, 0],
    label="create_surface_using_open3d_ball_pivoting_and_max_distance_between_points",
)
plotter.show()

Steps To Reproduce

1. download RGB_depth folder from the link link
2. download camera poses views.txt
3. create a python file at the same directory
4. run this code

Expected behavior i should a point cloud.

Environment (please complete the following information):

• Operating system: Windows 10 64-bit
• Python version: Python 3.7.11
• Open3D version: 0.13.0
• Is this remote workstation?: no
• How did you install Open3D?: conda

Additional context is there any constrains on the projection matrices or depth images ?

[theNded]

You can easily tell problem 1 is from the intrinsic from this error log. depth image size is (640 x 480), but got (-1 x -1) from intrinsic.

This can be solved by adding these two lines:

intrinsic.width = 640
intrinsic.height = 480

The second problem is with the depth image format. We expect uint16 depth with sufficient quantization precision, but your depth images are uint8, so after the intrinsic is fixed this shall still not work. Please be sure to deliver depth in uint16 with sufficient precision in your SfM pipeline (not simply a type conversion, but provide enough quantization).

[saurabheights]

I faced the same issue in running the example run_system.py. Another reason for this error is use of path_intrinsic relative path to dataset directory, i.e. path_dataset. Pass absolute path for path_intrinsic.