Algorithm documentation for Point-to-plane ICP

[tvercaut]

Checklist

• [X] I have searched for similar issues.
• [X] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for master branch).

My Question

I was trying to understand the approach used in Open3D to optimise the Point-to-plane ICP objective: http://www.open3d.org/docs/release/tutorial/pipelines/icp_registration.html#Point-to-plane-ICP

The documentation points only to [ChenAndMedioni1992] and [Rusinkiewicz2001]. In both references, the choice of optimiser (given a set of correspondences), seems to be left to the discretion of the reader. Non-linear optimisation with Levenberg-Marquardt is mentioned as an option and so is a single step linearised problem solution.

In OpenCV, it looks like a Gauss-Newton optimiser is used: https://docs.opencv.org/4.x/d7/dbe/kinfu_icp.html

In Open3D, after reading through the code, it seems that a one-step linearised problem is solved: https://github.com/isl-org/Open3D/blob/f1a0f3edd6b6a7d96352f43f83844111e8e9ea35/cpp/open3d/pipelines/registration/TransformationEstimation.cpp#L59-L90

I guess this is the same as a single step of Gauss-Newton but this wasn't entirely clear to me.

It would be great if additional information would be provided on the matter, either in the overall documentation or directly as comments in the source code for this function.

[tvercaut]

Similarly, the robust version of the Point-to-plane ICP version is discussed here: http://www.open3d.org/docs/release/tutorial/pipelines/robust_kernels.html#Point-to-plane-ICP-using-Robust-Kernels where a reference to an iteratively reweighted least-squares (IRLS) approach is made. However, reading through the code snippet above, I couldn't find an inner iteration loop and am thus inclined to think that only a single step of IRLS is being performed. Am I missing something?

[saurabheights]

@tvercaut The loop is written in C++ and not in python wrapper code registration_icp to ensure optimal performance. See https://github.com/isl-org/Open3D/blob/master/cpp/open3d/pipelines/registration/Registration.cpp#L149 for the loop code

[tvercaut]

Thanks @saurabheights , as far as I can see, the code you pointed to is the outer iteration loop of the ICP. My question refers to the inner loop, which would be called as part of estimation.ComputeTransformation in the case of robust ICP or point-to-plane ICP when the objective function given the corespondences is non-linear.

As per my first post here, there doesn't seem to be an inner iteration loop in Open3D and the approach to optimise the objective function within each step of the outer loop this is not documented.

[saurabheights]

@tvercaut No need to thank me. I misread your first message :D.

My math skills are quite rusty but

I guess this is the same as a single step of Gauss-Newton but this wasn't entirely clear to me.

is correct. It would be good if official maintainers of repository would confirm this.

For documentation - You might find this helpful. Slide 8-14 and 43 are highlights, but to me every page of this slide was gold :)

3_non-linear_optm-RANSAC_annotated.pdf

[saurabheights]

Regarding your question about IRLS. I dont see why there should be an inner loop at all. Correct me if wrong.

I skimmed https://web.archive.org/web/20221017041048if_/https://cnx.org/exports/92b90377-2b34-49e4-b26f-7fe572db78a1@12.pdf/iterative-reweighted-least-squares-12.pdf to review.

[tvercaut]

Iteratively reweighted least-squares (IRLS) is iterative and thus requires a loop as the name suggest and as detailled in the file you linked. That is unless a single step is considered sufficient in the context of the ICP since there is already an outer loop to establish correspondences.