Descriptors convertible to OpenCV?

[Karol-G]

Am I right to assume that data in SiftPoint contains a descriptor? Can I just take these arrays from all SiftPoints and convert them to an OpenCV descriptor Mat? Are they normalized to 0 to 255? Or do I need the values like scale, sharpness and so on in some way too?

I won't have time to build and test it myself the next few days, so it would be cool to know.

[mtee]

@CookingCookie have you ever found the solution?

The CudaSift descriptors cannot be interchanged with the opencv descriptors, since the opencv descriptor values are actual histogram counts, i.e. integer values. The CudaSift descriptors consists of float values between 0 and 1. I assume, that the difference is that these steps are applied in CudaSift:

Since there are 4 × 4 = 16 histograms each with 8 bins the vector has 128 elements. This vector is then normalized to unit length in order to enhance invariance to affine changes in illumination. To reduce the effects of non-linear illumination a threshold of 0.2 is applied and the vector is again normalized.

While in opencv, the resulting value is either upscaled again or the normalization simply does not happen. Same as you, I was looking for a quick solution for matching CudaSift features against OpenCV-Sift features, but I guess I will have to dive in deeper into code and the papers.

[isgursoy]

same here

[behzad-ost]

Has anyone found a solution for this? I also need to know what are the keypoints and descriptors from opencv sift in cudaSift. or anyway to convert them together maybe?

[Karol-G]

Yeah, I managed to solve this problem, but forgot to post the solution.

Here is a documented working example:

EasyGpuSift.h

#pragma once
#ifdef GPU
#include <cudaImage.h>
#include <cudaSift.h>

/// <summary>
/// An easy to use wrapper for the gpu sift feature detector.
/// </summary>
class EasyGpuSift
{
public:

    /// <summary>
    /// An easy to use wrapper for the gpu sift feature detector.
    /// </summary>
    EasyGpuSift();
    ~EasyGpuSift();

    /// <summary>
    /// This method is empty for gpu sift.
    /// Use compute(Mat& image, std::vector<KeyPoint>& keypoints, Mat& descriptors, int maxKeypoints) instead.
    /// Gpu sift cannot be split in detect and compute and can only use the TopN filter.
    /// </summary>
    /// <param name="image">The image.</param>
    /// <param name="keypoints">An empty list will be returned.</param>
    void detect(cv::Mat& image, std::vector<cv::KeyPoint>& keypoints);

    /// <summary>
    /// Computes the keypoints and descriptors in an image.
    /// Keypoints are always filtered with TopN and limited by the parameter maxKeypoints.
    /// Only the descriptors will be returned.
    /// </summary>
    /// <param name="image">The image.</param>
    /// <param name="descriptors">A reference to the computed descriptors that will be returned.</param>
    /// <param name="maxKeypoints">The maximum number of keypoints to detect. This parameter is only used by Gpu Sift.</param>
    void compute(cv::Mat& image, cv::Mat& descriptors, int maxKeypoints);

    /// <summary>
    /// Computes the keypoints and descriptors in an image.
    /// Keypoints are always filtered with TopN and limited by the parameter maxKeypoints.
    /// </summary>
    /// <param name="image">The image.</param>
    /// <param name="keypoints">A reference to the detected keypoints that will be returned.</param>
    /// <param name="descriptors">A reference to the computed descriptors that will be returned.</param>
    /// <param name="maxKeypoints">The maximum number of keypoints to detect. This parameter is only used by Gpu Sift.</param>
    void compute(cv::Mat& image, std::vector<cv::KeyPoint>& keypoints, cv::Mat& descriptors, int maxKeypoints);
};
#endif

EasyGpuSift.cpp

//#include "stdafx.h"
#include "EasyGpuSift.h"
#ifdef GPU

EasyGpuSift::EasyGpuSift()
{
}

EasyGpuSift::~EasyGpuSift()
{
}

void EasyGpuSift::detect(cv::Mat& image, std::vector<cv::KeyPoint>& keypoints)
{

}

void EasyGpuSift::compute(cv::Mat& image, cv::Mat& descriptors, int maxKeypoints)
{
    std::vector<cv::KeyPoint> keypoints;
    compute(image, keypoints, descriptors, maxKeypoints);
}

void EasyGpuSift::compute(cv::Mat& image, std::vector<cv::KeyPoint>& keypoints, cv::Mat& descriptors, int maxKeypoints)
{
    SiftData siftData;
    InitSiftData(siftData, maxKeypoints, true, true);

    cv::Mat tmpImage;
    cv::cvtColor(image, tmpImage, CV_RGB2GRAY);
    tmpImage.convertTo(tmpImage, CV_32FC1);
    CudaImage cudaImg;
    cudaImg.Allocate(image.size().width, image.size().height, iAlignUp(image.size().width, 128), false, NULL, (float*)tmpImage.data);
    cudaImg.Download();

    int numOctaves = 5;
    float initBlur = 1.0f;
    float thresh = 2.0f; //3.5f;
    float minScale = 0.0f;
    bool upScale = false;

    ExtractSift(siftData, cudaImg, numOctaves, initBlur, thresh, minScale, upScale);

    // Convert SiftData to Keypoints
    keypoints.resize(siftData.numPts);
    cv::parallel_for_(cv::Range(0, siftData.numPts), [&](const cv::Range& range) {
        for (int r = range.start; r < range.end; r++)
        {
            keypoints[r] = cv::KeyPoint(cv::Point2f(siftData.h_data[r].xpos, siftData.h_data[r].ypos), siftData.h_data[r].scale, siftData.h_data[r].orientation, siftData.h_data[r].score, siftData.h_data[r].subsampling, siftData.h_data[r].match);
        }
    });

    // Convert SiftData to Mat Descriptor
    std::vector<float> data;

    for (int i = 0; i < siftData.numPts; i++)
    {
        data.insert(data.end(), siftData.h_data[i].data, siftData.h_data[i].data + 128);
    }

    cv::Mat tempDescriptor(siftData.numPts, 128, CV_32FC1, &data[0]);
    //descriptors = tempDescriptor; // Buggy!
    tempDescriptor.copyTo(descriptors); // Inefficient!

    FreeSiftData(siftData);
}
#endif

[eudora-jia]

hello everyOne I CANT use this Opencv Sift codes with parallel ways, I am new one to cuda But I indeed check the .cu code, And I found the golbal variables `constant int d_MaxNumPoints; device unsigned int d_PointCounter[82+1]; constant float d_ScaleDownKernel[5]; constant float d_LowPassKernel[2LOWPASS_R+1]; constant float d_LaplaceKernel[812*16]; ` in cudaSiftD.cu Could anyone can help me get over it ?

[gloria111]

thanks for the code ,i tried the code in easygpu sift.cpp and transferd cuda sift to opencv description format successfully , than i found the values of description.row(1) are like 0.018337278, 0.00063770503, 2.5431522e-05, 1.7774635e-05, 0.00030450703, 0.0024855391, 0.023249455, ...while the same input images using opencv sift descriptor 's i get the description.row(1) valuse are 0, 0, 0, 86, 26,...... any suggestions?