Feature overall improves

[kalwalt]

Trying to improve the code and to make a real working example. I have made some remarkable progresses, but when the pinball image is tracked the blue rect corner is distorted, and iframe too. @EdwardLu2018 maybe you have an idea what i'm doing wrong here? Most of the code is based on your https://github.com/EdwardLu2018/wasm-ar trying to integrate it into WebARKitLib.

[kalwalt]

need to check this https://docs.opencv.org/4.5.0/dc/d16/tutorial_akaze_tracking.html

[kalwalt]

I'm testing this output in PR https://github.com/kalwalt/wasm-ar/pull/1 see comment https://github.com/kalwalt/wasm-ar/pull/1#discussion_r1131673222 and the comment after that one.

[kalwalt]

now with the last commit https://github.com/kalwalt/webarkit-testing/pull/7/commits/c05faa87be8aec63cab42a5571aa1c0dd02bac9c i got this result:

Reference image keypoints: 3043
WebARKit.js:2 Reference image descriptors: [61 x 3043]

compared with the result from wasm-ar:

WASM module loaded.
img_tracker_wasm.js:9 Reference image keypoints: 7516
img_tracker_wasm.js:9 Reference image descriptors: [61 x 7516]
img_tracker_wasm.js:9 Ready!

it is really a big difference. The wasm-ar code flipped the source image, and i did also this (flipping on the orizontal axes) , and image data from jpeg has 3 channels, i suppose RGB changed the conversion routine according to this. The number of keypoints/descriptors is increased but not as in wasm-ar. I will try to change the flipping and the channels types, if it is not sufficient i will try to implement the same code as in wasm-ar. I'm going to create also a new class for Akaze to separate things.

[kalwalt]

I refactored the C++ code wrapper inside WebARKitJS.cpp, it not export functions but a whole class: WebARKit. I think it is much better. As a consequence also the javascript code is changed in accord with this refactor. But when i run the app i got even less descriptors/Keypoints:

Orb created!
WebARKit.js:2 BFMatcher created!
WebARKit.js:2 refCols: 1637
WebARKit.js:2 refRows: 2048
WebARKit.js:2 Gray Image!
WebARKit.js:2 Reference image keypoints: 1736
WebARKit.js:2 Reference image descriptors: [32 x 1736]
WebARKit.js:2 Orb Detect and Compute passed!

[kalwalt]

i adapted my code using the wasm-ar code, that is converting the tracker image to grayscale with the GrayScaleMedia class, and provide gray video with the same class (GrayScaleMedia). modified also the html file structure, but same result the blue rect that should wrap the tracker image while tracking is completely distorted or is too little and not corresponding to the image tracked at all. I have the supsect that something is failing passing the data from the C++ code to JS. I will try to investigate this. Note: the descriptors are correctly created, this is almost sure, and most of the C++ code is correct.

[kalwalt]

Note also that AKAZE still fails with this error: WebARKitController.js:152 ERROR: RuntimeError: table index is out of bounds for testing, you need to change the tracker inside WebARKitController and rebuild.

[kalwalt]

with commit https://github.com/kalwalt/webarkit-testing/pull/7/commits/f80d07bbc12026372060ff208149280cdc98b4ab added an EM_ASM to outputting values the result is the same:

output 171.54806518554688 117.5586929321289 217.1331787109375 123.27236938476562 143.65428161621094 66.23845672607422 208.7754364013672 109.4506607055664
example.html:39 (8) [171.54806518554688, 117.5586929321289, 217.1331787109375, 123.27236938476562, 143.65428161621094, 66.23845672607422, 208.7754364013672, 109.4506607055664]0: 171.548065185546881: 117.55869293212892: 217.13317871093753: 123.272369384765624: 143.654281616210945: 66.238456726074226: 208.77543640136727: 109.4506607055664length: 8[[Prototype]]: Array(0)

[kalwalt]

With the latest commit i got this error in the console: webarkit_ES6_wasm.js:8 ==42==ERROR: AddressSanitizer: attempting double-free on 0x149c0800 in thread T0:, in specific note that is pointing to the initTrackerGray method and this is caused because i'm invoking a free of resources where they where already freed. The problem is in the WebARKitOrbTracker at this line:

L21  free(refData);

https://github.com/kalwalt/WebARKitLib/blob/9795d2708182f78f256344c3800b37b6277b6159/lib/SRC/WebARKitTrackers/WebARKitOpticalTracking/WebARKitOrbTracker.cpp#L21

and:

L59  free(refData);

https://github.com/kalwalt/WebARKitLib/blob/9795d2708182f78f256344c3800b37b6277b6159/lib/SRC/WebARKitTrackers/WebARKitOpticalTracking/WebARKitOrbTracker.cpp#L59

I will remove those lines and the same for the WebARKitAkazeTracker

gist log here

[kalwalt]

with commit the double-free issue was solved, i will leave the debug flags until i'm not sure that other issue will not happens.

[kalwalt]

While testing the Akaze tracker i got this error:

Start!
webarkit_ES6_wasm.js:8 8000
webarkit_ES6_wasm.js:8 Orb created!
webarkit_ES6_wasm.js:8 BFMatcher created!
webarkit_ES6_wasm.js:8 refCols: 1637
webarkit_ES6_wasm.js:8 refRows: 2048

webarkit_ES6_wasm.js:8 ==42==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x14cf3000 at pc 0x0000ce15 bp 0x12eadbf0 sp 0x12eadbfc

see that is printing Orb created (is hould print Akaze created)

Anyway is pointing as before the initTrackerGray and initialize_raw investigating on this.

gist here

[kalwalt]

Ok that issue was solved add now i have a better result while detecting! The iframe, when the marker is tracked, is displayed onto the pinball image, and the blue square is shifted, but i think it's a css issue. Probably we are almost readyto merge to master.

[kalwalt]

in wasm-ar the canvas and video is directly in the body, while in our example i'm creating a Container with a div element with the canvas and video inside; i will try the same setup.

[kalwalt]

I have exeperiencing some flashing and the blue rectangle is not completely aligned. Anyway i will remove the code that we non't neeed and after this PR can be merged.