Advanced options: load any model from HF or allow local models

[do-me]

Right now, the model from HF is hard-coded and supports only English. While it is just one line of code and one could easily recompile the project, it would be nicer to have advanced options for custom models, either from HF or local ones in the GUI e.g. to support other languages too.

However, usability shouldn't be sacrificed, so I'd stick to all-MiniLM-L6-v2 as default because it works great with English and is super small. Under the advanced options section there should be either a drop down or a combined input text field for a custom URL or for file uploads (local models). Not sure what's best though, but the latter seems more versatile and would clutter the UI less than having two different inputs, drop-down and input text box. But a drop-down would be more convenient for the average user and offer some space for notes (link to model explanation, supported languages etc.) maybe...

[varunneal]

This gets to a good point which will probably be my next task. WebML needs more comprehensive benchmarking and a better grasp on how "large" we can actually get in-browser. I'll benchmark some of the existing webml models in terms of memory performance and some canonical measurements for their task. Then I'll basically verify they work on various browsers (will probably test chrome, safari, mobile.). Hopefully I can make something resembling HF leaderboards: https://huggingface.co/spaces/optimum/llm-perf-leaderboard

I'd guess Xenova has a good internal intuition for how big is possible but the data should be more readily available.

[lizozom]

I started working on this repo as basis for an npm package that would allow anyone injecting their content and searching on it.

Something with an API along these lines

async init(modelName: string, modelConfig: ModelConfig) -> bool
async embedContent(content: string, config: EmbeddingConfig) -> EmbeddingMap 
async search(query: string, threshold: number, maxResults: number) -> Array<SearchResult> // This could also return an Observable and stream results as they are found

Does this align in any way with your plans? Do you envision something else for this repo?

[do-me]

That's a really cool idea to create an npm package with an easy-to-use API - I support this 100%!  Thinking about it, it sounds like the right way to go as well, decoupling the semantic search logic and the frontend. Also this would benefit the browser plugin idea.

I was already thinking about a more efficient logic considering the indexing part / vector storage. I pinged the folks from Qdrant here (initial question) and here (answer) whether they were interested in providing a WASM version of Qdrant for faster and more efficient vector storage / querying. They said it's not that easy but at some point they might give it a try, so no ETA yet. The reason why I was interested are all the optimizations that would make quite a difference for larger texts / more segments.

It would be interesting to port semantic.js to Rust, then compile to WASM and see whether there are speed and memory benefits. To my understanding there should be as Rust has more fine-grained control about data structures. I'm not an expert on Rust though, just what friends told me. :D

[lizozom]

I created the semantic-js library. It exposes the methods I mentioned - init, embedContent and search. Internally it uses a WebWorker and supports transformers.js threading.

I also created a version of SemanticFinder that uses it (it's currently in a branch, but I could merge it back into yours).

Let me know what you think!

[do-me]

That's pretty cool! Let me have a closer look at the library tomorrow, but as said earlier I do appreciate the modular idea :)

[do-me]

With the latest changes SemanticFinder supports all models tagged with feature-extraction on HF. TBH I don't know how frequent it would happen that people would really like to test their own models locally - there is always the option of hosting it on HF and adding an option in index.html like I did for do-me/test for testing some models. Closing for now but if there's still interest in using local models we might reopen any time.