Issues in RANSCA registration using 0.13.0 (& 0.14.1), no issue with 0.10.0

[chz31]

Dear all,

I originally used RANSAC and ICP in open3d 0.10.0 to register a group of mouse skulls, and it works fine. However, with the 0.13.0 and 0.14.1 version, the skulls are now aligned poorly (see the screenshots below). I have tested these versions of open3d on both Windows 10 and Linux with Python 3.6.7 and 3.8.10. I counter the same issues. I am pretty sure it is cause by RANSAC registration.

I submitted a similar question early on but replied late, so the issue was closed (#4154 https://github.com/isl-org/Open3D/issues/4154). Sorry for the delayed response. I did what you asked by increasing the iteration rounds to 4000,000 and confidence level to 0.99999, but it still does not work.

I've created python script with pilot data (posted on GitHub) to reproduce this issue. You may clone or download the repository as zip file that contains the data (mouse ply files) and scripts here: https://github.com/chz31/o3d_test . The "data" folder in the repository "o3d_test" contains the mouse models for you to test.

The scripts are in "test_scripts" folder. The "test_0.10.0.py" contains the script using open3d 0.10.0. The "test_0.13.0.py" contains script using open3d 0.13.0.

For each script, please modify three variables for accessing and output data (line 129 to 133 for test_0.10.0.py and line 131 to 135 for test_0.13.0.py): " sourceFileDir" points to the folder that contains the mouse models downloaded from the Github repo I provided above. "targetFileDir" points to a specific model "129_S1" in the "sourceFileDir" that serves as the universal target for registration. "outDir" points to the folder that stores the mouse models register to the target "129_S1"

To run the script after specifying the above three directories and path, you can simply run in your python console by using the code:

1. path = "Enter_Your_Directory_that_stores_the_script_for_o3d_0.10.0/test_0.10.0.py" #Specify the path to test o3d 0.10.0.py exec(open(path).read()) #The registered models will be stored in "outDir"

2. path = "Enter_Your_Directory_that_stores_the_script_for_o3d_0.10.0/test_0.13.0.py" #Specify the path to test o3d 0.13.0.py exec(open(path).read()) #The registered models will be stored in "outDir"

You can use the ransac_list variable to show the ransac transformation matrices.

Please let me know if you have any issues running the code or any further information.

I look forward to hearing from you. Thank you very much.

Chi

[theNded]

Can you please first try to change 4 to 3 here and see how it improves? https://github.com/chz31/o3d_test/blob/main/test_scripts/test_0.13.0.py#L81

[chz31]

Can you please first try to change 4 to 3 here and see how it improves? https://github.com/chz31/o3d_test/blob/main/test_scripts/test_0.13.0.py#L81

Thank you for your reply. Just tried it with both 0.13.0 and 0.14.1. Still got the same issue.

[theNded]

I can reproduce the problem, looking into it.

[theNded]

It requires some parameter tuning. The following snippet works with the command python registration.py /path/to/129S1_SVIMJ_.ply /path/to/PERC_EiJ_.ply --voxel_size 0.3. It consistently works for all the pairs, though there are minor shifts possibly due to the scanning distortion.

How to tune parameters? Here are some tricks:

• Voxel size matters. You were using 0.5, which is too large and smoothed out too many details. 0.3 works better.
• Distance threshold for RANSAC matters. The skull has many thin structures, so we get to be more tolerant for the fragile correspondences.
• Mutual filter (which was originally set False) can help but is not critical.

I will include the snippet in the examples. I will close this issue, but please feel free to follow up if other issues occur.

import open3d as o3d

import argparse

def preprocess_point_cloud(pcd, voxel_size):
    pcd_down = pcd.voxel_down_sample(voxel_size)
    pcd_down.estimate_normals(
        o3d.geometry.KDTreeSearchParamHybrid(radius=voxel_size * 2.0,
                                             max_nn=30))
    pcd_fpfh = o3d.pipelines.registration.compute_fpfh_feature(
        pcd_down,
        o3d.geometry.KDTreeSearchParamHybrid(radius=voxel_size * 5.0,
                                             max_nn=100))
    return (pcd_down, pcd_fpfh)

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('src', type=str, help='path to src point cloud')
    parser.add_argument('dst', type=str, help='path to dst point cloud')
    parser.add_argument('--voxel_size',
                        type=float,
                        default=0.05,
                        help='voxel size in meter')

    args = parser.parse_args()

    voxel_size = args.voxel_size
    ransac_distance_threshold = 3 * voxel_size
    icp_distance_threshold = 1.5 * voxel_size

    o3d.utility.set_verbosity_level(o3d.utility.VerbosityLevel.Debug)
    print('Reading inputs')
    src = o3d.io.read_point_cloud(args.src)
    dst = o3d.io.read_point_cloud(args.dst)

    print('Downsampling inputs')
    src_down, src_fpfh = preprocess_point_cloud(src, voxel_size)
    dst_down, dst_fpfh = preprocess_point_cloud(dst, voxel_size)

    print('Running RANSAC')
    result = o3d.pipelines.registration.registration_ransac_based_on_feature_matching(
        src_down, dst_down, src_fpfh, dst_fpfh, True, ransac_distance_threshold,
        o3d.pipelines.registration.TransformationEstimationPointToPoint(False),
        3, [
            o3d.pipelines.registration.CorrespondenceCheckerBasedOnEdgeLength(
                0.9),
            o3d.pipelines.registration.CorrespondenceCheckerBasedOnDistance(
                ransac_distance_threshold)
        ], o3d.pipelines.registration.RANSACConvergenceCriteria(1000000, 0.999))

    src_down.paint_uniform_color([1, 0, 0])
    dst_down.paint_uniform_color([0, 1, 0])

    print('Running ICP')
    src_down.estimate_normals()
    dst_down.estimate_normals()
    result = o3d.pipelines.registration.registration_icp(
        src_down, dst_down, icp_distance_threshold, result.transformation)
    o3d.visualization.draw(
        [src_down.transform(result.transformation), dst_down])

Example outputs: 129S1_SVIMJ vs NOD_SHILTJ ARK_J_SKULL vs BTBR_T_xxx NOD_SHILTJ vs PERC_EIJ NZW_LACJ vs 129S1_SVIMJ

[chz31]

@theNded Thank you very much! Your suggestions are really helpful.

Just a follow up, I used your settings with my code to align all 61 mouse models I have. Only one specimen shows misalignment. I further decrease the voxel size to 0.25, and it appears everything is good now.

Could you please let me know if there are any significant difference in the algorithm between 0.10.0 and 0.13.0, just in case if we encounter issues in the future? The RANSAC registration works so well and stable in O3D 0.10.0 for the same sample with voxel size is around 0.5, and I feel it still yields slightly better results than 0.13.0.

We are using open3d to align models to a template to automatically place landmarks on them. Therefore, registration is really critical to our method.

Thank you!

[theNded]

To be honest I am also confused because technically only parallelization and the approach to compute the exit condition have been changed. In the trials, we exhausted all the iterations so the latter is not an issue. I cannot give you an answer since I don't have time to visit all the details of your implementation, but I will keep it in mind and update you if I observe anything suspicious later.

[chz31]

I see. Thank you very much again for your help!

[theNded]

Problem identified. This function was called incorrectly. This function should be used instead.

I will be addressing this issue in a following PR and afterwards you can switch back to your code.

[chz31]

Thank you for letting me know!

[chz31]

Hi, I saw that the open3d RANSAC issues appear to be fixed and this issue has been closed. Could you please let me know when the new version with the fixes will be accessible by "pip install"? I ask this because we are using open3d for a module in the 3D Slicer software platform. Open3d 0.10.0 might not work with the next update of Slicer. Thank you very much!