search

Omniplex-ai / omniplex

Open-Source Perplexity
https://omniplex.ai
GNU Affero General Public License v3.0
869 stars 159 forks source link

Ollama support #6

Open ImVexed opened 6 months ago

ImVexed commented 6 months ago

Would be nice to support self hosted LLMs. It doesn't have to be ollama, but it seems to be fairly easy to interface with.

jank commented 4 months ago

I have quite limited knowledge on TS or yarn. By poking around I came to the following minimal changes required to talk to a local LLM hosted with Ollama (in my case a Gemma 2B).

https://github.com/Omniplex-ai/omniplex/compare/main...jank:omniplex:main

Please note that you need a an OpenAI key and Bing key in your environment. OpenAI Key is required as GPT-3 is used for the 'tool' selection process (e.g. if to use search or stocks or weather). Replacing that by Ollama will likely require a fine-tuned prompt.

Also note that I had to comment out the Firebase Authentication. Otherwise the web app would not correctly start.