Global registration and ICP usually give incorrect values

[trinhnhut-gif]

---

Describe the bug Hi, I am trying to match two point cloud data. One is a simulation point cloud by CloudCompare and another one is a real point cloud by VLP-16 lidar. I used global registration and ICP to match but sometimes the result is not correct and sometimes is correct, I don't know is it a bug or just my code really don't work in my case, but I will be thanksfull if you can help me. Thanks!

To Reproduce Steps to reproduce the behavior: Environment (please complete the following information):

Operating system:Ubuntu 18.04 Python version: python 3.8.6 Open3D version: 0.12 Is this remote workstation?: no How did you install Open3D?: pip

Expected behavior I tried to change the threshold and also the voxel size but still not solve my problem

Screenshots Attached

Environment (please complete the following information):

• Operating system: Windows 10 64-bit
• Python version: Python 3.8
• Open3D version: 0.13.0
• How did you install Open3D?: pip

Additional context import open3d as o3d import numpy as np import matplotlib.pyplot as plt from mpl_toolkits import mplot3d import copy import os

def preprocess_point_cloud(pcd, voxel_size): print(":: Downsample with a voxel size %.3f." % voxel_size) pcd_down = pcd.voxel_down_sample(voxel_size) pcd_down, ind = pcd_down.remove_statistical_outlier(nb_neighbors=20, std_ratio=2.0)

radius_normal =   voxel_size * 3
print(":: Estimate normal with search radius %.3f." % radius_normal)
pcd_down.estimate_normals(
    o3d.geometry.KDTreeSearchParamHybrid(radius=radius_normal, max_nn=30))
# orient normals so the ground is facing the same way
pcd_down.orient_normals_to_align_with_direction()
pcd_down = remove_ground_points(pcd_down, 0.4)

radius_feature =  voxel_size *5
print(":: Compute FPFH feature with search radius %.3f." % radius_feature)
pcd_fpfh = o3d.pipelines.registration.compute_fpfh_feature(
    pcd_down,
    o3d.geometry.KDTreeSearchParamHybrid(radius=radius_feature, max_nn=30))
return pcd_down, pcd_fpfh

def remove_ground_points(pcd, z_threshold):

pop_list = []
for i, normal in enumerate(pcd.normals):
    if normal[2] > z_threshold:
        pop_list.append(i)

print("Number of points to be removed: ", len(pop_list))
pop_list.sort(reverse=True)
for item in pop_list:
    pcd.normals.pop(item)
    pcd.points.pop(item)

return pcd

def prepare_dataset(voxel_size): print(":: Load two point clouds") source = o3d.io.read_point_cloud("A.pcd") target = o3d.io.read_point_cloud("B.pcd") trans_init = np.asarray([[0.0, 0.0, 1.0, 0.0], [1.0, 0.0, 0.0, 0.0], [0.0, 1.0, 0.0, 0.0], [0.0, 0.0, 0.0, 1.0]])

draw_registration_result(source, target, np.identity(4))

source_down, source_fpfh = preprocess_point_cloud(source, voxel_size)
target_down, target_fpfh = preprocess_point_cloud(target, voxel_size)
return source, target, source_down, target_down, source_fpfh, target_fpfh

def draw_registration_result(source, target, transformation): source_temp = copy.deepcopy(source) target_temp = copy.deepcopy(target) source_temp.paint_uniform_color([1, 0.706, 0]) target_temp.paint_uniform_color([0, 0.651, 0.929]) source_temp.transform(transformation) o3d.visualization.draw_geometries([source_temp, target_temp], zoom=0.4559, front=[0.6452, -0.3036, -0.7011], lookat=[1.9892, 2.0208, 1.8945], up=[-0.2779, -0.9482, 0.1556])

def execute_global_registration(source_down, target_down, source_fpfh, target_fpfh, voxel_size): distance_threshold = 10 print(":: RANSAC registration on downsampled point clouds.") print(" Since the downsampling voxel size is %.3f," % voxel_size) print(" we use a liberal distance threshold %.3f." % distance_threshold) result = o3d.pipelines.registration.registration_ransac_based_on_feature_matching( source_down, target_down, source_fpfh, target_fpfh, True, distance_threshold, o3d.pipelines.registration.TransformationEstimationPointToPoint(False), 3, [ o3d.pipelines.registration.CorrespondenceCheckerBasedOnEdgeLength( 0.9), o3d.pipelines.registration.CorrespondenceCheckerBasedOnDistance( distance_threshold) ], o3d.pipelines.registration.RANSACConvergenceCriteria(10000000, 0.999)) return result

def refine_registration(source, target, source_fpfh, target_fpfh, voxel_size): distance_threshold = voxel_size print(":: Point-to-point ICP registration is applied on original point") print(" clouds to refine the alignment. This time we use a strict") print(" distance threshold %.3f." % distance_threshold)

result = o3d.pipelines.registration.registration_icp(
    source, target, distance_threshold, result_ransac.transformation,
    o3d.pipelines.registration.TransformationEstimationPointToPoint(),
    o3d.pipelines.registration.ICPConvergenceCriteria(max_iteration=2000))
return result

if name == "main":

voxel_size = 3
source, target, source_down, target_down, source_fpfh, target_fpfh = prepare_dataset(
voxel_size)
result_ransac = execute_global_registration(source_down, target_down,
                                        source_fpfh, target_fpfh,
                                        voxel_size)
print(result_ransac)

draw_registration_result(source, target, result_ransac.transformation)

print(result_ransac.transformation)

result_icp = refine_registration(source, target, source_fpfh, target_fpfh,
                                voxel_size)
draw_registration_result(source, target, result_icp.transformation)

[theNded]

Global registration is not guaranteed to be working. On real-world dataset the success rate is around 10%-30%. It is usually refined in pose graph optimization where many observation constraints are applied.

For your case, failure is especially more likely when you register two point clouds with different point distributions. Apparently, the real-world scan has a sparser distribution, while the simulation is denser. This will result in different FPFH features (and even more advanced convolutional deep features), therefore causing very few correspondences. Removing ground points further reduces the potential correspondences, and leads to the failure of registration.

I personally think this is still an open issue and will be happy to implement it if you find relevant references.

[beyondmz]

I also encountered the same problem, which is to register with my own point cloud data. How did you register in the end

[Sumeers]

Hello, I am having similar issue, I am trying to register lidar point cloud data from vehicle with point cloud data generated from citygml lod2 data. While doing global registration, the initial and final output are same, there is no registration conducted. RegistrationResult with fitness=0.000000e+00, inlier_rmse=0.000000e+00, and correspondence_set size of 0 Which means there is no registration, if any one has similar issue and solution please help.