Use E5-multilingual as a default

snexus / llm-search

Querying local documents, powered by LLM

MIT License

424 stars 50 forks source link

Use E5-multilingual as a default #18

Closed Logophoman closed 11 months ago

Logophoman commented 12 months ago

Instruct-xl indeed is on rank 2 atm, while E5-multilingual is on place 5, however multilingualism will greatly enhance the overall usability of the document search imo, it performs really well when used.

I guess just make it optional by default, just as the LLM, but I suggest using that as a default embedding model. It's also only 2.2 Gb not ~5GB large...

Great Repo!

snexus commented 12 months ago

Thanks for the suggestion, I will add a support for it.

snexus commented 11 months ago

Added support for e5 models through sentence-transformers. Great model indeed - subjectively pulling more relevant results compared to the instructor-large. Thanks again!