SACSegmentationFromNormals returning worst fit when setting setOptimizeCoefficients to true

[DuffRumkins]

I have been working on a script which uses pcl::SACSegmentationFromNormalsto fit a cylinder model to segments in a point cloud.

For this I had had the setOptimizeCoefficients to true since, as I understood it, this would result in better model coefficients for the resulting fit. I noticed some issues when trying to fit a roughly cylindrical section of a point cloud representing a branch. The axis of the cylinder looked correct, but the radius was negative. I had no idea how this was possible as other segments were being fit correctly.

After some messing about, I found that changing setOptimizeCoefficients to false fixed this issue and the branch was able to be fit quite well.

Am I misunderstanding how this optimisation works or is this an issue with implementation? Surely a negative radius should never be possible?

Thanks for your help!

[mvieth]

Hi! The optimization is done via an unconstrained Levenberg-Marquardt approach. During the optimization, the radius is used squared (like radius*radius), so a positive radius and a negative radius with the same absolute value would lead to the same result. See here: https://github.com/PointCloudLibrary/pcl/blob/master/sample_consensus/include/pcl/sample_consensus/sac_model_cylinder.h#L300

Things you could check:

• the inliers returned by segment, are they better or worse when you optimize vs don't optimize?
• if you use the absolute value of the radius returned by segment, does everything make sense then?

In general I would agree that the radius should not be negative. I will further look into the implementation when I find time. If you are willing to share your point cloud and code (snippets), I might be able to tell you more about your specific case.

[DuffRumkins]

Hi mvieth, thanks for checking this out!

Below are the model coefficients for the branch segment when optimisation is set to false:

Model coefficients for cluster 0 are 
 header: 
seq: 0 stamp: 0 frame_id: camera_frame
values[]
  values[0]:   0.0860656
  values[1]:   0.0712902
  values[2]:   0.74954
  values[3]:   -0.937655
  values[4]:   0.0470675
  values[5]:   0.344366
  values[6]:   0.0395515

When optimisation is set to true, I get the following coefficients:

Model coefficients for cluster 0 are 
 header: 
seq: 0 stamp: 0 frame_id: camera_frame
values[]
  values[0]:   0.00769405
  values[1]:   0.0754249
  values[2]:   0.822651
  values[3]:   -0.935314
  values[4]:   0.0344959
  values[5]:   0.352134
  values[6]:   -0.00632992

I also included a screenshot of the good fit (optimisation is false) and you can see that it does fit the cluster well. The second value is less than 1cm and would not be a good fit even if I took the absolute value.

It might also be important to note that I am using the PCL version that is supported with ROS noetic (1.10, I believe).

Finally, here is a snippet of the segmentation code I am using:

    pcl::SACSegmentationFromNormals<PointT, PointN> seg;    
   // Changing this from false to true is what causes the issue
    seg.setOptimizeCoefficients (optimise_cylinder);
    // the segmentation object is now set to look for an object of type SACMODEL_CYLINDER
    seg.setModelType (pcl::SACMODEL_CYLINDER);
    // sample consensus segmentation method used is SAC_RANSAC
    seg.setMethodType (pcl::SAC_RANSAC);
    //NormalDistanceWeight is set to 0.1
    seg.setNormalDistanceWeight (cylinder_normal_distance);
    // Maximum iterations used is 10000
    seg.setMaxIterations (cylinder_max_iterations);
    // Distance threshold used (for inliers) is 0.01m
    seg.setDistanceThreshold (cylinder_inlier_distance);
    // Cylinder must have a radius between 0.026m and 0.05m
    seg.setRadiusLimits (cylinder_radius_min, cylinder_radius_max);
    seg.setInputCloud (cluster_cloud);
    seg.setInputNormals (cluster_normals);
   seg.segment (*inliers_cylinder, *coefficients_cylinder);

Let me know if you need anything else!

[mvieth]

It might also be important to note that I am using the PCL version that is supported with ROS noetic (1.10, I believe).

You can try the latest PCL version of course, but I currently can't think of any changes to the relevant code since version 1.10. So PCL 1.10 or PCL 1.13 should not make a difference for this problem.

Let me know if you need anything else!

For me to reproduce the issue, I would need your point cloud as a PCD or PLY file (if you are willing and allowed to share it)

What you can also try is turning on debug output by calling pcl::console::setVerbosityLevel(pcl::console::L_VERBOSE); before segmenting with optimization. Then it should print a message beginning with [pcl::SampleConsensusModelCylinder::optimizeModelCoefficients] LM solver .... That might give us more information on what went wrong. BTW, did you check if segment reports more or fewer inliers if you turn optimization on?

[DuffRumkins]

Here is the PCD file of the branch.

Below is the debug output for the optimisation.

[pcl::RandomSampleConsensus::computeModel] Trial 852 out of 851.855205: 7 inliers (best is: 21 so far) (thread 0).
[pcl::RandomSampleConsensus::computeModel] Model: 2 size, 21 inliers.
[pcl::SampleConsensusModelCylinder::optimizeModelCoefficients] LM solver finished with exit code 2, having a residual norm of 8.97831e-06. 
Initial solution: 0.0860656 0.0712902 0.74954 -0.937655 0.0470675 0.344366 0.0395515 
Final solution: 0.00769405 0.0754249 0.822651 -1.3192 0.0486544 0.496663 -0.00632992

When I count the number of inliers using inliers_cylinder->indices.size() I get that there are 0 inliers which is different that what the debug output says.

Counting again without optimisation, I get 21 inliers which looks like it was the best solution anyway.

Any ideas?

[mvieth]

Exit code 2 would translate to RelativeErrorTooSmall (see LevenbergMarquardt.h in the Eigen project), so the algorithm seems to have converged. The residual norm is also quite small. You could try again with a smaller distance threshold, maybe 0.005? That's all I can say for now, I will come back when I have more time.

[DuffRumkins]

Decreasing the distance threshold to 0.005 does result in the optimised radius no longer being negative, but does not fix the issue.

The radius is still much too small, now even smaller at 0.00371741.

This is also way below the lower radius limit value so maybe this is not getting passed through correctly?

[mvieth]

@DuffRumkins I finally found some time to look into this again. The optimizeModelCoefficients function does indeed not check if the optimised coefficients fulfil the specified model constraints, but it should. However constrained optimisation would be much more complex to implement, so the alternative would probably be to just return the original coefficients (so in your specific situation you would not win much with this addition, except being sure that the results of optimizeModelCoefficients are not worse than before).

I also had a look at the branch_cluster pointcloud. To be honest, I don't see any curvature around the branch axis, so it does not surprise me that the optimisation fails. If you get acceptable results when turning the optimisation off, I would just go with that.

[DuffRumkins]

Thanks for taking the time to investigate this issue!

I think for now I will just keep optimizeModelCoefficients to false as this has been working well for me.

Thanks again!