search

norlab-ulaval / libpointmatcher

An Iterative Closest Point (ICP) library for 2D and 3D mapping in Robotics
BSD 3-Clause "New" or "Revised" License
1.61k stars 544 forks source link

Different transformation matrix at every execution #491

Open siddas27 opened 2 years ago

siddas27 commented 2 years ago

I am getting different values with large differences every time I execute the file. Is there anything I am doing wrong?

YoshuaNava commented 2 years ago

Hi @siddas27, Two questions:

  1. Which file are you executing?
  2. Can you paste some of the values that you are getting?
siddas27 commented 2 years ago

I am using the icp_advance_api.py code in examples/python with default.yaml on my own dataset. Here are the transformation matrix values after each execution - 

[[ 0.23931442 -0.17084752 -0.95579267  2.5380044 ]
 [-0.9698527   0.0045525  -0.24364842  1.2903088 ]
 [ 0.04597787  0.9852872  -0.1646074   1.7219288 ]
 [ 0.          0.          0.          1.        ]]
match ratio: 0.950093

------------------
Execution time : 0.43399500846862793
bag
[[ 0.77277046 -0.409994    0.48449028  0.04608268]
 [-0.04762023  0.7237487   0.68841904  0.00100982]
 [-0.63289666 -0.55506146  0.5397675  -0.05243272]
 [ 0.          0.          0.          1.        ]]
match ratio: 0.950061

------------------
Execution time : 0.23326754570007324
bag
[[ 0.5658166  -0.7645672   0.30868873  0.19490582]
 [ 0.51653314  0.6204949   0.5900672   0.18167305]
 [-0.6426857  -0.17442201  0.7460106  -0.26445025]
 [ 0.          0.          0.          1.        ]]
match ratio: 0.950037
pomerlef commented 2 years ago

@aguenette , this seems to be from the Python binding. Any idea?

aguenette commented 2 years ago

@pomerlef I don't why it's doing this, but I've got the same behavior on my side as well. I tried to rebuild the bindings after pulling the latest changes on the master branch and I wasn't able to import the module anymore.

Then, I checked out the commit (99d2937) when we merge the bindings into master and I was able to import the module again AND the problem of having different results at each execution was gone. I'm getting consistent results.

So, I guess something has been broken since then and I don't know what. It will require some more investigation to find what happened and since when.

@siddas27 Can you checkout the commit 99d2937, rebuild the bindings and verified that it solves the problem? Can you also gives us more information about your configurations and which Python version did you use to build the bindings?

Thanks!

boxanm commented 2 years ago

@aguenette The c0d13ea commit with bindings to Python3.8 built yesterday gave me consistent results on the example dataset. @siddas27 The default.yaml configuration uses a Random sampling filter, isn't your issue caused by having a different input point cloud on each execution?

aguenette commented 2 years ago

@boxanm Thanks for your help! I realized that I was building the bindings with pointmatcher set as a static library, so that's why I wasn't able to import the python library, it couldn't find the shared library. Everything's ok now.

Also, you are right about the RandomSampling filter, it has changed since the time I did the bindings, so that was not a behavior I encountered almost 2 years ago.

Now, the RandomSamplingFilter use the C++11 random library to generate random numbers, but before it was using C std::rand.

So, @siddas27 I would say that it's a normal behavior now.

Here are some results as an example:

On the master branch:

# Execute from the build directory
# .examples/icp_advance_api ../data/car_cloud400.csv ../data/car_cloud401.csv

# First run
match ratio: 0.850013

------------------
Haussdorff distance: 21.625 m
Haussdorff quantile distance: 0.32634 m
Robust mean distance: 0.0935902 m
------------------

ICP transformation:
  0.981613   0.169668 -0.0874597  0.0728684
 -0.152805   0.973057   0.172658   0.190908
  0.114398  -0.156119   0.981091 -0.0342262
         0          0          0          1

# Second run
match ratio: 0.850034

------------------
Haussdorff distance: 21.6181 m
Haussdorff quantile distance: 0.324561 m
Robust mean distance: 0.0943196 m
------------------

ICP transformation:
  0.981689   0.169818 -0.0863088   0.065078
 -0.153195   0.973086   0.172148   0.192254
   0.11322  -0.155774   0.981283  -0.035326
         0          0          0          1

# Third run
match ratio: 0.850016

------------------
Haussdorff distance: 21.601 m
Haussdorff quantile distance: 0.32597 m
Robust mean distance: 0.0933052 m
------------------

ICP transformation:
  0.981597   0.170031 -0.0869248  0.0716563
 -0.153239   0.972973   0.172749   0.191294
  0.113948   -0.15625   0.981123 -0.0340109
         0          0          0          1

# from the examples directory (need to install the library or to move it into this directory from the build/python directory)
# python3 icp_advance_api

# First run
match ratio: 0.75005979

------------------
Haussdorff distance: 21.5839 m
Haussdorff quantile distance: 0.327589 m
Robust mean distance: 0.0709974 m
------------------

3D ICP transformation:
   0.98149588  0.17015992 -0.08781479  0.08364291 
  -0.15315218  0.97287308  0.17338505  0.19156804 
   0.11493583 -0.15672768  0.98093129 -0.02654031 
   0.          0.          0.          1.        

# Second run
match ratio: 0.75003978

------------------
Haussdorff distance: 21.5939 m
Haussdorff quantile distance: 0.327172 m
Robust mean distance: 0.0713101 m
------------------

3D ICP transformation:
   0.98148068  0.17014484 -0.08801371  0.07863176 
  -0.15305848  0.97281602  0.17378746  0.18852621 
   0.11519018 -0.15709779  0.98084224 -0.02672877 
   0.          0.          0.          1.       

# Third run
match ratio: 0.75001986

------------------
Haussdorff distance: 21.5782 m
Haussdorff quantile distance: 0.326335 m
Robust mean distance: 0.0712839 m
------------------

3D ICP transformation:
   0.98148135  0.17022335 -0.08785431  0.08024962 
  -0.15325454  0.97292646  0.17299467  0.19169791 
   0.11492352 -0.15632697  0.98099667 -0.02698822 
   0.          0.          0.          1.

And on 99d2937:

# from the build directory
# .examples/icp_advance_api ../data/car_cloud400.csv ../car_cloud401.csv

# First run
match ratio: 0.850048

------------------
Haussdorff distance: 21.5966 m
Haussdorff quantile distance: 0.325087 m
Robust mean distance: 0.0934736 m
------------------

ICP transformation:
  0.981596   0.170035 -0.0869334  0.0732947
 -0.153272   0.973012   0.172496   0.192628
  0.113918  -0.155997   0.981166 -0.0353611
         0          0          0          1

# Second run
match ratio: 0.850048

------------------
Haussdorff distance: 21.5966 m
Haussdorff quantile distance: 0.325087 m
Robust mean distance: 0.0934736 m
------------------

ICP transformation:
  0.981596   0.170035 -0.0869334  0.0732947
 -0.153272   0.973012   0.172496   0.192628
  0.113918  -0.155997   0.981166 -0.0353611
         0          0          0          1

# Third run
match ratio: 0.850048

------------------
Haussdorff distance: 21.5966 m
Haussdorff quantile distance: 0.325087 m
Robust mean distance: 0.0934736 m
------------------

ICP transformation:
  0.981596   0.170035 -0.0869334  0.0732947
 -0.153272   0.973012   0.172496   0.192628
  0.113918  -0.155997   0.981166 -0.0353611
         0          0          0          1

# from the examples directory (need to install the library or to move it into this directory from the build/python directory)
# python3 icp_advance_api

# First run
match ratio: 0.75003948

------------------
Haussdorff distance: 21.5431 m
Haussdorff quantile distance: 0.326925 m
Robust mean distance: 0.0707997 m
------------------

3D ICP transformation:
   0.98137763  0.1707142  -0.08806028  0.08385201 
  -0.1536492   0.97277418  0.17350022  0.18948691 
   0.11528171 -0.15673884  0.98088891 -0.02664683 
   0.          0.          0.          1.          

# Second run
match ratio: 0.75003948

------------------
Haussdorff distance: 21.5431 m
Haussdorff quantile distance: 0.326925 m
Robust mean distance: 0.0707997 m
------------------

3D ICP transformation:
   0.98137763  0.1707142  -0.08806028  0.08385201 
  -0.1536492   0.97277418  0.17350022  0.18948691 
   0.11528171 -0.15673884  0.98088891 -0.02664683 
   0.          0.          0.          1.        

# Third run
match ratio: 0.75003948

------------------
Haussdorff distance: 21.5431 m
Haussdorff quantile distance: 0.326925 m
Robust mean distance: 0.0707997 m
------------------

3D ICP transformation:
   0.98137763  0.1707142  -0.08806028  0.08385201 
  -0.1536492   0.97277418  0.17350022  0.18948691 
   0.11528171 -0.15673884  0.98088891 -0.02664683 
   0.          0.          0.          1.