Open kalwalt opened 2 years ago
Speaking about sorting the keypoints, OpenCV use nor qsort
or sort
but anohter sorting algorithm, probably because more efficient ? KeyPoint
OpenCV class and Keypoint_t
from jsfeat are almost similar, and so probably we can adapt the code from OpenCV to our needs. If we look the Orb code in ComputeKeypoints method keypoints are removed those close to the border and then filtered:
// Remove keypoints very close to the border
KeyPointsFilter::runByImageBorder(keypoints, img.size(), edgeThreshold);
// Keep more points than necessary as FAST does not give amazing corners
KeyPointsFilter::retainBest(keypoints, scoreType == ORB_Impl::HARRIS_SCORE ? 2 * featuresNum : featuresNum);
this is the retainBest method:
// takes keypoints and culls them by the response
void KeyPointsFilter::retainBest(std::vector<KeyPoint>& keypoints, int n_points)
{
//this is only necessary if the keypoints size is greater than the number of desired points.
if( n_points >= 0 && keypoints.size() > (size_t)n_points )
{
if (n_points==0)
{
keypoints.clear();
return;
}
//first use nth element to partition the keypoints into the best and worst.
std::nth_element(keypoints.begin(), keypoints.begin() + n_points - 1, keypoints.end(), KeypointResponseGreater());
//this is the boundary response, and in the case of FAST may be ambiguous
float ambiguous_response = keypoints[n_points - 1].response;
//use std::partition to grab all of the keypoints with the boundary response.
std::vector<KeyPoint>::const_iterator new_end =
std::partition(keypoints.begin() + n_points, keypoints.end(),
KeypointResponseGreaterThanOrEqualToThreshold(ambiguous_response));
//resize the keypoints, given this new end point. nth_element and partition reordered the points inplace
keypoints.resize(new_end - keypoints.begin());
}
}
https://github.com/opencv/opencv/blob/64aad34cb4abfb6a2603f3f4ecae7f4f0ba1414d/modules/features2d/src/keypoint.cpp#L68-L90 May we implement this inside webarkit-jsfeat-cpp? we have to try!
while testing with docker and emsdk:3.1.25 i got this error and of course fails to build:
/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/WebARKitLib/lib/SRC/KPM/FreakMatcher/framework/image.h:89:37: error: ISO C++17 does not allow dynamic exception specifications [-Wdynamic-exception-spec]
size_t channels) throw(Exception);
^~~~~~~~~~~~~~~~
/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/WebARKitLib/lib/SRC/KPM/FreakMatcher/framework/image.h:89:37: note: use 'noexcept(false)' instead
size_t channels) throw(Exception);
^~~~~~~~~~~~~~~~
noexcept(false)
1 error generated.
emcc: error: '/home/kalwalt/emsdk/upstream/bin/clang++ -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -DEMSCRIPTEN -Werror=implicit-function-declaration -I/home/kalwalt/emsdk/upstream/emscripten/cache/sysroot/include/SDL --sysroot=/home/kalwalt/emsdk/upstream/emscripten/cache/sysroot -Xclang -iwithsysroot/include/compat -I/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/WebARKitLib/include -I/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/build/ -I/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/ -I/home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/WebARKitLib/lib/SRC/KPM/FreakMatcher -Oz -D HAVE_NFT /home/kalwalt/kalwalt-github/webarkit-jsfeat-cpp/emscripten/WebARKitLib/lib/SRC/KPM/FreakMatcher/detectors/DoG_scale_invariant_detector.cpp -c -o /tmp/emscripten_temp_qwriigpt/DoG_scale_invariant_detector_94.o' failed (returned 1)
exec error: 1
Built at Tue Nov 22 19:10:49 CET 2022
Done!
Need to open a issue on webarkitlib in the meanwhile i will continue to use emsdk 3.1.20
Somewhere the code is broken, infact in the Yape06 detect
function the compute_laplacian function print anything (just put a cout to print the dst
). It means that or tsome of the args are empty or there is a memory leak. Probably can be heplful to add some specific flags for debugging.
I discovered that i missed an important step: the pattern image imported by ar2ReadJpegImage it should be converted to grayscale (this is the missed step) but i can't apply because all the methods are specific for emscripten but not to be used internally. Before all i have to make the imgproc methods more flexible and rearrange the code a bit. I will do this in another specific PR.
Ok with the latest commit https://github.com/kalwalt/webarkit-jsfeat-cpp/pull/31/commits/6fe0b12be30cfd0ae214a38981e187de3cc78264 we recieve a count
number from detect_keypoints
. I will make some comments in the code to use as future reference. The main problem was to adapt javascript code to C++. See the while
cicle and you can understand what i mean.
Things to do:
unique_ptr
as tried before.At this point of the code we got this from te output log:
jsfeatES6cpp_debug.js:1667 JSLOG [info] Level 0 with 3352576 keypoints.
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1667 JSLOG [info] Level 1 with 1676288 keypoints.
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1667 JSLOG [info] Level 2 with 838144 keypoints.
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1667 JSLOG [info] Level 3 with 419072 keypoints.
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1655 Size of first lev_corners: 3352576
jsfeatES6cpp_debug.js:1667 JSLOG [info] After Gaussian blur
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1667 JSLOG [debug] deleting data_t
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1655 Count inside Yape06 detect_internal: 64192
jsfeatES6cpp_debug.js:1667 JSLOG [debug] deleting data_t
put_char @ jsfeatES6cpp_debug.js:1667
jsfeatES6cpp_debug.js:1655 Count inside detect_keypoints: 64192
jsfeatES6cpp_debug.js:1667 JSLOG [info] train 1637 x 2048 points: 300
instead from jsfeatNext:
train 512x384 points: 300
sample_orb.html:195 train 362x271 points: 300
sample_orb.html:195 train 255x191 points: 261
sample_orb.html:195 train 181x135 points: 139
what is wrong with the c++ code?
JSLOG [info] train 1637 x 2048 points: 300
display only one line because we only print for the first corners level. i will fix this.
with commit https://github.com/kalwalt/webarkit-jsfeat-cpp/pull/31/commits/4a0cf972479da4792f106c1acbe71ca5d145bd64 i added the training for the other levels, but it doesn't correctly calculate. I think there is an issue with the resample method, not sure if it ever worked correctly.
In this Pull request i will try do develop some utilities to train a Orb pattern. Partially following the jsfeat code but also OpenCV implementation as it was the original one.