How to 3d reconstruction with .npy files? (output)

[ParkJS99]

thank you for great project. i can't find any guide or info about 3d reconstruction with droid-slam's output. can i get some help?

this is what i tried

import open3d as o3d
import numpy as np
import quaternion

data_path = 'reconstructions/recon_test'

disps = np.load(data_path + '/disps.npy')
images = np.load(data_path + '/images.npy')
intrinsics = np.load(data_path + '/intrinsics.npy')
poses = np.load(data_path + '/poses.npy')

# depth scale factor
depth_scale = 1 / 256

point_cloud = o3d.geometry.PointCloud()

u, v = np.meshgrid(range(images.shape[3]), range(images.shape[2]))
x = (u - intrinsics[0, 2]) * disps / intrinsics[0, 0] * depth_scale
y = (v - intrinsics[1, 2]) * disps / intrinsics[1, 1] * depth_scale
z = disps * depth_scale

points = np.vstack((x.flatten(), y.flatten(), z.flatten(), np.ones_like(x.flatten()))).T

colors = images[:, [2, 1, 0], :, :].transpose(0, 2, 3, 1).reshape(-1, 3) / 255

point_cloud.points = o3d.utility.Vector3dVector(points[:, :3])
point_cloud.colors = o3d.utility.Vector3dVector(colors)

for i in range(poses.shape[0]):
    # homogeneous transform matrix로 변환
    pose_matrix = np.eye(4)
    pose_matrix[:3, :3] = quaternion.as_rotation_matrix(quaternion.from_float_array(poses[i, 3:]))
    pose_matrix[:3, 3] = poses[i, :3]

    # point_cloud 객체의 위치 및 방향 설정
    point_cloud.transform(pose_matrix)

o3d.visualization.draw_geometries([point_cloud])

[ParkJS99]

solved by save pcd at visualization.py

[yec22]

@ParkJS99 hi, can you provide the code to save pcd at visualization.py, thanks a lot!

[ParkJS99]

@ParkJS99 hi, can you provide the code to save pcd at visualization.py, thanks a lot!

save the numpy array at animation_callback function. (pts and clr)

like this

global pts_result, clr_result pts_result = np.append(pts_result, pts, axis=0) clr_result = np.append(clr_result, clr, axis=0)

============================== end point

result_point_actor = create_point_actor(pts_result, clr_result) o3d.io.write_point_cloud(f'pcd_result.pcd', result_point_actor)