search

Filimoa / open-parse

Improved file parsing for LLM’s
https://filimoa.github.io/open-parse/
MIT License
2.42k stars 93 forks source link

Different Embedding Models #8

Open gvlx opened 6 months ago

gvlx commented 6 months ago

Your code example seems to imply that.

Filimoa commented 6 months ago

Currrently yes - we're shipping support for open source embeddings very soon!

tan-yong-sheng commented 3 months ago

Hi @Filimoa,

I have commented here: https://github.com/Filimoa/open-parse/issues/10#issuecomment-2214343987, but after that, I found here could be a better place to put this comment:

I would like to suggest adding support for Litellm.

Litellm is an open source project, which unifies the API call of 100+ LLMs (including Anthropic, Cohere, and Ollama, etc) in an OpenAI compatible format: https://github.com/BerriAI/litellm

I believe integrating Litellm would be a fantastic enhancement because people could choose to switch or use their preferred embedding model api instead of OpenAI's ones only when dealing with semantic processing. Thanks.

For example, if they used litellm python client and without self hosting litellm proxy, then their code could be like this (which is very consistent with OpenAI python client format):

image

Reference: https://github.com/BerriAI/litellm

if someone self hosted litellm proxy, which they can call LLM API in an OpenAI compatible format via llmlite proxy, you could see the code could be as follows:

image

Reference: https://litellm.vercel.app/docs/providers/azure_ai#passing-additional-params---max_tokens-temperature

There are also quite a few projects that used litellm: https://litellm.vercel.app/docs/project to call models from different providers on LiteLLM.