DMatch .imdIdx(), .queryIdx(), and .trainIdx() methods fail silently

[BrydonLeonard]

Trying to run any of these to access the indeces after matching with a FlannBasedMatcher fail silently and causes my app to crash. I've tried stepping through the code and putting it in a try/catch, but I'm having trouble finding any information.

I have an arrayList keyPoints of KeyPointVectors, from which I get the specific key point vector. goodMatches is a DMatchVector. I've separated what I'm trying to do into multiple lines to isolate the issue. It crashes on line 3 of this code:

                KeyPointVector kpv = keyPoints.get(pointIndex);
                DMatch match = goodMatches.get(i);
                int idx = match.queryIdx();
                KeyPoint kp = kpv.get(idx);
                Point2f pt = kp.pt();

[saudet]

What indexes are you trying to access and what is the size of those vectors?

[BrydonLeonard]

I'm not sure if I understand the question correctly, but keyPoints has a size of 3, the size of each KeyPointVector is different, I populate them using a SURF object's .detect method.

It's my understanding that the queryIdx() should give me the position of the matched keypoint in terms of the Mat I'm trying to match from, and the .trainIdx() should give me the index of that same keypoint in terms of the scene.

I couldn't find a javaCV example doing this exactly, so I've been trying to use this openCV tutorial to work from.

[saudet]

Ok, so, for example, how do you allocate KeyPointVector?

[BrydonLeonard]

This is, I think the relevant code: I get a number of KeyPoints and Descriptors first, and put them in the keyPoints ArrayList, keyPoints = new ArrayList<KeyPointVector>(); and the following is run to add new KeyPoints

            KeyPointVector pointKeyPoints = new KeyPointVector(50);
            Mat pointDescriptors = new Mat();
            surf.detect(pointMat, pointKeyPoints);
            surf.compute(pointMat, pointKeyPoints, pointDescriptors);

            keyPoints.add(pointKeyPoints);
            descriptors.add(pointDescriptors);

I then have this "find" method, to match one of the previously detected objects, in a new scene. The new scene is passed in as "image", and "pointIndex" is the index of the specific object to find, in terms of its index in the keyPoints and descriptors lists.

public Point find(Mat image, int pointIndex) {
        KeyPointVector sceneKeyPoints = new KeyPointVector();
        Mat sceneDescriptors = new Mat();
        surf.detect(image, sceneKeyPoints);
        surf.compute(image, sceneKeyPoints, sceneDescriptors);

        IndexParams indexParams = new IndexParams();
        indexParams.setAlgorithm(0);

        SearchParams searchParams = new SearchParams(50);
        FlannBasedMatcher matcher = new FlannBasedMatcher(indexParams, searchParams);

        DMatchVectorVector matches = new DMatchVectorVector();
        matcher.knnMatch(descriptors.get(pointIndex), sceneDescriptors, matches, 2);

        long size = matches.size();

        DMatchVector goodMatches = new DMatchVector();
        for (long i = 0; i < size; i++) {
            if (matches.get(i).size() >= 2) {
                DMatch match1 = matches.get(i).get(0);
                DMatch match2 = matches.get(i).get(1);

                if (match1.distance() <= NNDR_RATIO * match2.distance()) {
                    goodMatches.put(match1);
                }
            }
        }

        long goodSize = goodMatches.size();
        Point2fVector objCoords = new Point2fVector(goodSize);
        Point2fVector sceneCoords = new Point2fVector(goodSize);
        for (long i = 0; i < goodSize; i++) {
            try {
                KeyPointVector kpv = keyPoints.get(pointIndex);
                DMatch match = goodMatches.get(i);
                int idx = match.queryIdx();
                KeyPoint kp = kpv.get(idx);
                Point2f pt = kp.pt();
                objCoords.put(pt);
                sceneCoords.put(sceneKeyPoints.get(goodMatches.get(i).trainIdx()).pt());
            } catch (Exception e) {
                e.printStackTrace();
            }
        }

        Mat H = opencv_calib3d.findHomography(new Mat(objCoords), new Mat(sceneCoords));

        DoubleIndexer h = H.createIndexer();

        double x = POINT_SIZE / 2;
        double y = POINT_SIZE / 2;
        double z = 1 / (h.get(6) * x + h.get(7) * y + h.get(8));
        double px = h.get(0) * x + h.get(1) * y + h.get(2) * z;
        double py = h.get(3) * x + h.get(4) + h.get(5) * z;

        return new Point((int)px, (int)py);
    }

[saudet]

Ok, so, for example, what are you trying to do with the following line?

objCoords.put(pt);

[BrydonLeonard]

What I'm attempting to do is have a Point2fVector representing keypoints in object space (objCoords) and one representing the points in scene space (sceneCoords) and use findHomography so that I can map a point from object space into scene space.

My intention with that line was to add the coordinates of one of the matched key points (in object space) to the vector of key points representing coordinates in object space.

[saudet]

I understand your intentions, I'm just pointing you to code that's pretty much guaranteed to cause memory corruption.

[BrydonLeonard]

I'm not really sure how to solve that issue. Does it have to do with the Point2fVector data structure? The other issue is that the app is crashing 3 lines before that, so unfortunately I don't think that's the main problem at the moment.

[saudet]

Right, so try to find errors like that one in the rest of the code, fix them, and see if it runs better.

[BrydonLeonard]

Sorry, but I am still lost as to how I should correct the lines like that one. I may be missing something fundamental here.

I also fixed my log filters, so I can now see that I'm getting Fatal signal 11 (SIGSEGV), code 1, fault addr 0x991e0880 in tid 17740 (Thread-13759).

[saudet]

http://bytedeco.org/javacpp/apidocs/org/bytedeco/javacpp/Pointer.html#put-org.bytedeco.javacpp.Pointer-

[saudet]

Just don't call that method or any method from the Pointer class, period, and that should fix all your issues.