Closed varunneal closed 1 year ago
According to MTEB, I'll probably go ahead and convert instructor-large and probably e5-small-v2 to ONNX weights and host them on HF sometime soon.
Thanks a lot for this awesome feature, that's some great work! I'll have a look at your benchmarks later, I'm very curious about the results.
Before merging, I think I found two little things we should correct first:
I just tested e.g. the multilingual model with French, Italian and German and indeed mostly the results are quite good (for such a small model of only 135Mb!).
Yet, e.g. in this example with Italian, white spaces for some reason generally have a high similarity score and mess with the results:
Thanks for catching these bugs! Will work on them. I'll get back to you when I
Top-k is a great idea for replacing the default of 0.3!
By the way, would you mind adding public viewing permissions for your spreadsheet?
Yes! I haven't populated it much yet. To measure chrome memory I'm using Task Manager as an approximation. Here's a JS function to measure server-side memory usage:
function getTotalMemoryUsage() {
const memoryUsage = process.memoryUsage();
return memoryUsage.heapUsed + memoryUsage.external + memoryUsage.arrayBuffers;
}
I'm using process.cpuUsage();
to measure inference time.
@VarunNSrivastava maybe you could split this PR into two? Doing the UI changes and the embedding changes separately?
Because splitting the package out could be a useful approach for this, but we're going to get a ton of conflicts at the moment.
I thought about it some more, feel free to go ahead and merge the PR! Reintroducing the package is pretty easy!
Please share your thoughts on https://github.com/do-me/SemanticFinder/issues/24 though, as it would change my implementation!
Unfortunately, the UI addition of the progress bar necessitated that I use WebWorkers for model loading and embedding, so it won't be very easy to separate the two.
@do-me The latest couple commits should fix a lot of the problems.
@lizozom It should be easy to load the embeddings right at the start, though as mentioned, it could be a very slow operation at the start if the text is large.
Great work @VarunNSrivastava - I'm particularly excited about testing more languages and see how well they perform!
Thank you for the amazing work @VarunNSrivastava ! I'll work on revising my PR.
As discussed, we should have an option to load any model. This feature is now in advanced settings. Additionally, drawing from Xenova's transformers.js demo, we have a loading bar corresponding to downloading the models. This requires using workers, which involved some code refactoring.
As always, the demo for this PR is live at https://varunnsrivastava.github.io/SemanticFinder/.
A few key points: