search

c0sogi / LLMChat

A full-stack Webui implementation of Large Language model, such as ChatGPT or LLaMA.
MIT License
245 stars 40 forks source link

Universal Sentence Encoder for embedding #38

Closed Torhamilton closed 11 months ago

Torhamilton commented 1 year ago

Openai embedding is quite weak, proprietary and token hungry. Lets move to USE. PDFGPT has an implementation of USE we can modify and build upon.

c0sogi commented 1 year ago

The project seems like that it uses scikit KNN retriever. Qdrant's cosine similarity search will be more powerful than KNN. The main feature i can get from it is how I get the embedding vectors.

I already implemented Transformer embedding, which uses embedding model from Huggingface. This requires pytorch library, and your link requires tensorflow. I will add both as seperate modules and add some requirements.txt files.

BTW, both local embeddings require a lot of computing resources. Make sure your environment is strong enough. I tested intfloat/e5-large-v2 and it almost freezed my 13600K, handling 20 pages of PDF.