surface_matching PPF3DDetector::match fails in samplePCByQuantization

[ahundt]

I've run into a case where ppf_helpers.cpp samplePCByQuantization fails because

const float* point = (float*)(&pc.data[i * pc.step]);

On line 278 the value contained in point aka *point contains the value NaN. It almost looks like a pointer is being multiplied by i, which seems quite strange, unless cv::MatStep is convertible into something else.

Here is the stack trace:

#0  0x00000001006de39f in std::__1::vector<int, std::__1::allocator<int> >::push_back(int const&) [inlined] at /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/vector:1587
#1  0x00000001006de39f in cv::ppf_match_3d::samplePCByQuantization(cv::Mat, float*, float*, float*, float, int) at /Users/athundt/source/git/Itseez/opencv_contrib/modules/surface_matching/src/ppf_helpers.cpp:288
#2  0x0000000100745a61 in cv::ppf_match_3d::PPF3DDetector::match(cv::Mat const&, std::__1::vector<cv::Ptr<cv::ppf_match_3d::Pose3D>, std::__1::allocator<cv::Ptr<cv::ppf_match_3d::Pose3D> > >&, double, double) at /Users/athundt/source/git/Itseez/opencv_contrib/modules/surface_matching/src/ppf_match_3d.cpp:499
#3  0x000000010000d06f in main at /Users/athundt/source/robonetracker/modules/grl/test/ppf_load_match.cpp:110
#4  0x00007fff8a2435c9 in start ()
#5  0x00007fff8a2435c9 in start ()

Here is the state of the relevant variables:

pc  cv::Mat     
flags   int 1124024325  1124024325
dims    int 2   2
rows    int 179279  179279
cols    int 9   9
data    uchar * 0x10d700020 0x000000010d700020
datastart   const uchar *   0x10d700020 0x000000010d700020
dataend const uchar *   ""  0x000000010dd27b3c
datalimit   const uchar *   ""  0x000000010dd27b3c
allocator   cv::MatAllocator *  NULL    0x0000000000000000
u   cv::UMatData *  0x10d4c9a30 0x000000010d4c9a30
size    cv::MatSize     
step    cv::MatStep     
p   size_t *    0x7fff5fbfdc08  0x00007fff5fbfdc08
*p  size_t  36  36
buf size_t [2]      
[0] size_t  36  36
[1] size_t  4   4
xrange  float * 0x7fff5fbfe3f0  0x00007fff5fbfe3f0
yrange  float * 0x7fff5fbfe3e8  0x00007fff5fbfe3e8
zrange  float * 0x7fff5fbfe3e0  0x00007fff5fbfe3e0
sampleStep  float   0.0500000007    0.0500000007
weightByCenter  int 0   0
map std::__1::vector<std::__1::vector<int, std::__1::allocator<int> >, std::__1::allocator<std::__1::vector<int, std::__1::allocator<int> > > > size=8000   
numSamplesDim   int 19  19
xr  float   254.999969  254.999969
yr  float   256 256
zr  float   254.999969  254.999969
numPoints   int 0   0
c   int 32767   32767
pcSampled   cv::Mat     
i   int 1   1
point   const float *   0x10d700044 0x000000010d700044
*point  const float NaN NaN
xCell   const int   -2147483648 -2147483648
yCell   const int   0   0
zCell   const int   0   0
index   const int   -2147483648 -2147483648

Clearly *point isn't right and neither is index.

[RayceRossum]

I am also receiving this error when trying to match. The method executes correctly when training the detector.

[ahundt]

@RayceRossum off topic but I'm now using https://github.com/tum-mvp/ObjRecRANSAC, plus some fixes at: https://github.com/ahundt/ObjRecRANSAC.

[RayceRossum]

Thank you @ahundt, I noticed this has also been implemented in the PCL library which I have been working with so I'll give that a try!

[tolgabirdal]

Could you provide some details on that or a sample code to reproduce the error?

[ahundt]

I think all my tests are in this folder: https://github.com/ahundt/grl/tree/master/test

It has been so long that I don't recall exactly what I did, but the main files should be fairly easy to use, sorry!

[feliwir]

I had this bug aswell. It happens when data in the mat is nullptr. Solution is to do a check before calling the match function, but this should really be done within the match function

[bmagyar]

@feliwir Could you please open a pull request with data and sample code to reproduce it and the fix itself? I'm sure many would appreciate it.

[feliwir]

@bmagyar @tolgabirdal Problem is line 286 and following in ppf_helpers.cpp:

  float xr = xrange[1] - xrange[0];
  float yr = yrange[1] - yrange[0];
  float zr = zrange[1] - zrange[0];

  int numPoints = 0;

  map.resize((numSamplesDim+1)*(numSamplesDim+1)*(numSamplesDim+1));

  // OpenMP might seem like a good idea, but it didn't speed this up for me
  //#pragma omp parallel for
  for (int i=0; i<pc.rows; i++)
  {
    const float* point = pc.ptr<float>(i);

    // quantize a point
    const int xCell =(int) ((float)numSamplesDim*(point[0]-xrange[0])/xr);
    const int yCell =(int) ((float)numSamplesDim*(point[1]-yrange[0])/yr);
    const int zCell =(int) ((float)numSamplesDim*(point[2]-zrange[0])/zr);
    const int index = xCell*numSamplesDim*numSamplesDim+yCell*numSamplesDim+zCell;

    /*#pragma omp critical
        {*/
    map[index].push_back(i);

When xr,yr or zr are 0 there occurs a division by 0. This happens for me with zr, so the zCell is completly messed up and the index is wrong aswell after that. I recommend a check for zero division, but i am not sure to which value xCell,yCell and zCell should be set in that special case

[Techtony96]

I am also having this issue, has anyone found a good solution for this? Just realized this was from 2017.

[Techtony96]

It looks like this issue was fixed in OpenCV 3.4.2 in this commit: https://github.com/opencv/opencv_contrib/commit/71191e36d6b278e1d20007b6b217d799388e84ae#diff-30596ba439a137919cd3407e960b71ea

I am going to test now and see if the issue is fixed.