search

BBC-Esq / VectorDB-Plugin-for-LM-Studio

Plugin that lets you use LM Studio to ask questions about your documents including audio and video files.
247 stars 34 forks source link

possibly use ctranslate2 and/or infinity embeddings #143

Closed BBC-Esq closed 4 months ago

BBC-Esq commented 4 months ago

Faster embeddings via Infinity from well-known ctranslate2 expert:

https://github.com/michaelfeil/infinity

BBC-Esq commented 4 months ago

Another option is to use ctranslate2 directly to create the embeddings, and if I switch to faiss, they can be directly input into the vector database. Would simply need to encode a query and make sure the faiss database gets it correctly along with any necessary parameters in order to conduct a search...basically looking for ways to speed up database creation and/or search.

base script:

https://github.com/michaelfeil/infinity/blob/main/libs/infinity_emb/infinity_emb/transformer/embedder/ct2.py