slow loading with multiple nfts

[albjeremias]

Hi, i've been using this code for a cordova app which uses multiple nft markers. I would like to improve the loading of .fset; .iset; .fset3;

But first I'm trying to understand where is the bottleneck and which solutions there are.

It seems the bottleneck is when calling "addFreakFeaturesAndDescriptors"

• https://github.com/webarkit/artoolkit5/blob/master/lib/SRC/KPM/kpmMatching.cpp#L296
• https://github.com/webarkit/artoolkit5/blob/master/lib/SRC/KPM/FreakMatcher/facade/visual_database_facade.cpp#L76

Possible solutions:

1. "we need to find a specific compression method for the feature sets, improved for mobile devices; this would make the loading of the 3 files (.fset .iset .fset3) faster" https://www.kalwaltart.com/blog/2020/01/21/nft-natural-feature-tracking-with-jsartoolkit5/

2. Why not store the the whole db of the images in a file, instead of the fset of each marker? (https://github.com/webarkit/artoolkit5/blob/master/lib/SRC/KPM/FreakMatcher/facade/visual_database_facade.cpp#L44)

Thanks for your efforts, amazing stuff :)

[kalwalt]

Hi @albjeremias Daniel @Carnaux and Thorsten @ThorstenBux already did such compression system, I think that it should be almost ready. :slightly_smiling_face:

[albjeremias]

hei, is there anyway a branch for testing this? Or trying to create a PR from it? thanks

[kalwalt]

@albjeremias it is a work done by @Carnaux and @ThorstenBux started the discussion in this issue on my repository https://github.com/kalwalt/jsartoolkit5/issues/34 the code should be in the Daniel jsartoolkit5 repository compression branch https://github.com/Carnaux/jsartoolkit5/tree/compression he developed also a tool to compress them https://github.com/Carnaux/ZFT-Creator

[albjeremias]

As you mentioned in https://github.com/kalwalt/jsartoolkit5/issues/34

takes minimum 420 ms and maximum 507 ms on my device. With the old approach ( loading .fset3 .iset .fset) instead it takes minimun 537 and maximum 719 ms.

I don't think that this is a very huge impact, was there any research on just trying to dump the part of the memory with memcpy to store the whole structure which jsartoolkit5 needs in the local storage of the browser, so that next time the user loads the page it won't take as long to load?

[albjeremias]

@kalwalt Do you think this approach might work? I have not much experience with emscript..

something like: https://stackoverflow.com/questions/54617194/how-to-save-files-from-c-to-browser-storage-with-emscripten#54627719 seems very tempting..

[kalwalt]

@albjeremias honestly i don't know, for sure to load the descriptors we need to call the C++ functions https://github.com/webarkit/jsartoolkit5/blob/4ed7e4b127f2b79cb4a19cac2266c29c06be69cf/emscripten/ARToolKitJS.cpp#L480. The article explain how to load a file with the FS provided by Emsrcipten, but since you need to call from the EM_ASM macro that is javascript code. If we found the system we can try, but it seems a bit difficult, Maybe @ThorstenBux or @Carnaux has a better idea on this regard.

[albjeremias]

Ok, I've been researching a bit more.. and found this very interesting wasm project: https://github.com/dfinity-side-projects/wasm-persist I'll try it on the next weeks... but if you peepz have some opinion on it would be helpful :) before wasting away my energy!

[kalwalt]

@albjeremias I agree with you it it sounds interesting, but I have no opinion on the subject. Don't think to waste your time, think to conquest more knowledge!

[kalwalt]

@albjeremias if you are interested PR https://github.com/webarkit/jsartoolkit5/pull/11 is on the way. :smile:

[kalwalt]

11 is merged now in the master branch!

[albjeremias]

Yesss!!! :)) im so greatuful super nice, see things going further! congratzz!

this should be aggregated to the webarkit: https://github.com/Carnaux/ZFT-Creator ;) also, maybe there should be different functions for loading compressed nfts.. just for compatibility sake, for other users to be able to still enjoyment improvements without the need to regenerate all nfts..?! or if its too much for maintenance, make sure to follow the version change to a bold number! ;)

[Carnaux]

If I'm not mistaken i added code to check if there is a zft or iset file. If there is no zft it loads the iset.

[albjeremias]

I get this error:

hexStrIset.match(...) is null artoolkit_wasm.js:1:25983
    loadZFT http://localhost:8080/artoolkit/artoolkit_wasm.js:1
    onSuccessZFT http://localhost:8080/artoolkit/artoolkit_wasm.js:1
    writeByteArrayToFS http://localhost:8080/artoolkit/artoolkit_wasm.js:1
    onload http://localhost:8080/artoolkit/artoolkit_wasm.js:1

[Carnaux]

@albjeremias When does that error happen? It seems that the compressed string from the iset file is empty. I can look at that later. In the meantime, you can use the creator tool to convert your iset, fset, and fset3 files to zft, or create a zft from scratch.

[albjeremias]

@albjeremias When does that error happen? It seems that the compressed string from the iset file is empty. I can look at that later. In the meantime, you can use the creator tool to convert your iset, fset, and fset3 files to zft, or create a zft from scratch.

i need a command line tool.. i have several nfts... https://github.com/Carnaux/NFT-Marker-Creator It seems the commandline might work...

node app.js -zft

but how do i specify the iset, fset and fset3 files?

[albjeremias]

@albjeremias When does that error happen? It seems that the compressed string from the iset file is empty. I can look at that later. In the meantime, you can use the creator tool to convert your iset, fset, and fset3 files to zft, or create a zft from scratch.

it says in there: loadZFT http://localhost:8080/artoolkit/artoolkit_wasm.js:1

and my files are not zft... so i guess thats the issue :)