[FPCSInitialAlignment] The output aligned point cloud is unstable

[QiuYilin]

The Aligned souce cloud output by FPCSInitialAlignment sometimes turns into a strange point cloud.

I am testing the FPCS coarse registration method，In this process, the transformation matrix and the aligned point cloud are very strange.

My code:
#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>
#include <pcl/registration/ia_fpcs.h>
#include <pcl/visualization/pcl_visualizer.h>

#include <iostream>

int main() {
  // create shared pointers
  pcl::PointCloud<pcl::PointXYZ> cloud_source_ptr, cloud_target_ptr;

  pcl::io::loadPCDFile<pcl::PointXYZ>("D:/program_on_git/own/test/pcl_test/source/bun0.pcd", cloud_source_ptr);
  pcl::io::loadPCDFile<pcl::PointXYZ>("D:/program_on_git/own/test/pcl_test/source/bun4.pcd", cloud_target_ptr);

  // initialize fpcs
  pcl::PointCloud<pcl::PointXYZ> source_aligned;
  pcl::registration::FPCSInitialAlignment<pcl::PointXYZ, pcl::PointXYZ> fpcs_ia;
  fpcs_ia.setInputSource(cloud_source_ptr.makeShared());
  fpcs_ia.setInputTarget(cloud_target_ptr.makeShared());

  fpcs_ia.setNumberOfThreads(10);
  fpcs_ia.setApproxOverlap(0.9);
  fpcs_ia.setDelta(1);
  fpcs_ia.setNumberOfSamples(8);

  // align
  std::cout << "Aligning..." << std::endl;
  fpcs_ia.align(source_aligned);

  auto transformation = fpcs_ia.getFinalTransformation();
  std::cout << "Transformation Matrix:\n" << transformation << std::endl;

  // visualize source_aligned
  pcl::visualization::PCLVisualizer viewer("Source Aligned");
  viewer.addPointCloud<pcl::PointXYZ>(source_aligned.makeShared(),
                                      "source_aligned");
  viewer.spin();

  return 0;
}

The point cloud file is provided in pcl/test.

case 0:

Transformation Matrix: -0.0247219 -0.263157 0.11795 0.0375599 -0.0799325 -0.850856 0.381365 0.154063 0.0171766 0.18284 -0.0819511 0.0312225 0 0 0 1

cloud:

case 1: Transformation Matrix: -0.154957 0.950659 -0.268769 -0.0808333 0.804335 -0.0365597 -0.59305 0.112859 -0.573614 -0.308078 -0.758983 0.0803555 0 0 0 1

image

case2: A matrix that is not documented but is not the identity matrix

image: A deformed point cloud, not a straight line nor a rabbit

At present, I have not discovered any rules from the phenomena.

Your Environment (please complete the following information):

OS: Windows 10 Compiler: MSVC 2022 PCL Version 1.14.1.99(https://github.com/PointCloudLibrary/pcl/commit/2d5101a59b75551f5300b78586b5521d006aa5cf) GPU:1060 6GB

[mvieth]

The method randomly selects points, and the random seed is based on the current time, so the results may be different every time the program is run. Of course, what you describe above should not happen. I have found a probable bug in linkMatchWithBase (a function used in the fpcs class), which results in the correspondences sometimes not being one-to-one. I will have to check what the best solution is.

[mvieth]

While I am still investigating several things in the fpcs algorithm, I have some questions regarding the parameters you set:

fpcs_ia.setDelta(1);
fpcs_ia.setNumberOfSamples(8);

Why did you choose these values? Did you have a specific intention, like making the algorithm more precise, or faster, or something else? Or did you copy these values from somewhere else? Answers to these questions might help me understand how the class should handle certain parameter choices.

[QiuYilin]

@mvieth they are from https://github.com/PointCloudLibrary/pcl/blob/master/test/registration/test_fpcs_ia_data.h

If I set numberofsamples to 100, there will be a exception.

And the output matrices are not the identity matrix, I made a mistake, I have updated the question.

The deformed point cloud other than the straight line did not appear later.

[QiuYilin]

I found that the source code test used setDelta(1.0,true)，which would not cause any problems. However, I used setDelta(1); which would cause an exception.

[mvieth]

What is the exception? Can you post a backtrace so that I can see where it comes from? setDelta(1) seems to be a bad choice because it is not adjusted to the given point cloud. setDelta(1, true) means that it is adapted to the density of the target point cloud.

[QiuYilin]

According to my recent implementation, the program will break without throwing an exception.The program cannot be closed normally, and forcibly closing it will give a memory overflow exception. A delta that is too large will cause all intermediate checks to pass, and it may be better to directly resolve the situation where the delta is too large during the compute initialization process.

[mvieth]

According to my recent implementation, the program will break without throwing an exception.The program cannot be closed normally, and forcibly closing it will give a memory overflow exception.

I am not sure I understand. Do you mean that the program does not finish on its own? Would it perhaps finish if you wait long enough?

A delta that is too large will cause all intermediate checks to pass, and it may be better to directly resolve the situation where the delta is too large during the compute initialization process.

What do you propose? How could the class determine whether the delta is too large?

[QiuYilin]

I am not sure I understand. Do you mean that the program does not finish on its own? Would it perhaps finish if you wait long enough?

After waiting long enough, the terminal shows that the program has ended, but there is no result output, so I don't continue to wait. Then closing the entire IDE will cause a memory exception. Could this be a multi-threading problem?

What do you propose? How could the class determine whether the delta is too large?

The delta in absolute mode and delta*mean_dist in relative mode should not exceed half of the maximum points pair distance of the target point cloud. I have not considered this limitation clearly. If this is a long wait caused by a large amount of calculation, it should not be considered as a bug.setDelta(1,true) should have satisfied most cases. At present, the problem of extreme cases is not important.