search

cpietsch / vikus-viewer-script

Scripts to generate sprite sheets and textures for VIKUS Viewer
https://vikusviewer.fh-potsdam.de/
11 stars 9 forks source link

Add ability for the TSNE script to use the tfjs-node library #13

Closed digitaldogsbody closed 2 years ago

digitaldogsbody commented 2 years ago

When running the TSNE script, the @tensorflow/tfjs library prints a message to console suggesting that the @tensorflow/tfjs-node library is a much faster alternative. This library is already in the package.json, but is not called in the code. I observed that it does not work on all systems (e.g. on Alpine Linux due to differences between musl and glibc that are not fixable with the compatibility layer), so perhaps this is why?

Testing a straight swap of libraries showed a speed improvement of over 30x - based on 43 input images and using time node tsne.js -i /data/vikus/images/input/ for measurement, it went from 2m32s to 5s.

This PR adds the ability to switch to the @tensorflow/tfjs-node library by passing a flag to the script -t. I left the original @tensorflow/tfjs library as the default to ensure that this would not break existing deployments.

cpietsch commented 2 years ago

Hey @digitaldogsbody, thanks for looking into that. Indeed the tfjs-node or even tfjs-node with gpu is much faster. I left it in there because locally I always used the tfjs-node module, but there were quite some compatibility problems back then. Adding the selection of tfjs via a flag is great!

digitaldogsbody commented 2 years ago

Thanks for the super quick response and improving my crappy javascript! Is there any chance you could add a tag / do a release with these and your other changes? I am pulling the .tar.gz down as part of a Docker image build, so it would be really helpful for me.

cpietsch commented 2 years ago

Sure thing, here it is https://github.com/cpietsch/vikus-viewer-script/releases/tag/v2.0.4