do-me
commented
5 months ago Thanks for the PR! The faster dot product is perfect no problem with that.
Tbh I don't see any speed difference to the current implenation, even with ~150k embeddings:
- https://varunnsrivastava.github.io/SemanticFinder/?hf=Collection_of_100_books_dd80b04b
- vs. https://do-me.github.io/SemanticFinder/?hf=Collection_of_100_books_dd80b04b
I was convinced that I already cached the queryEmbedding magnitude and length but apparently I did not implement this yet. However this could be another improvement and tickle some ms out of the distance calculation. In the current implentation this calculation is repeated by the number of chunks and hence causing lots of unnecessary calculations. In practice however, I guess it doesn't make that much of a difference.
See the variables here in the old function: https://github.com/do-me/SemanticFinder/pull/52/commits/e3897d9820748a09219cc737ed75c6ce71dcec18#diff-5dc8b349c945c9be3db790816ede2973f5609ab10e659bc1b45e67a3184818c9L300
As mentioned here https://github.com/do-me/SemanticFinder/issues/49, the Xenova dot product is much faster now. Check it out for yourself here: https://varunnsrivastava.github.io/SemanticFinder/