search

khoj-ai / khoj

Your AI second brain. Get answers to your questions, whether they be online or in your own notes. Use online AI models (e.g gpt4) or private, local LLMs (e.g llama3). Self-host locally or use our cloud instance. Access from Obsidian, Emacs, Desktop app, Web or Whatsapp.
https://khoj.dev
GNU Affero General Public License v3.0
12.37k stars 621 forks source link

What is "model" in the configuration for? #267

Closed wenlzhang closed 1 year ago

wenlzhang commented 1 year ago

In the configuration file, there are the following settings: chat-model: gpt-3.5-turbo model: text-davinci-003

I tried to use gpt-3.5-turbo for model but got some warning messages in the terminal. Now I am using text-embedding-ada-002, and I am not getting any errors.

Therefore, I am wondering what is this for? How to set this parameter to reduce costs?

sabaimran commented 1 year ago

~Hey @wenlzhang , could you specify which warnings you were getting? gpt-3.5-turbo should certainly work same as the others. The warning is most likely nothing to be worried about.~

Sorry, I misunderstood the original question. gpt-3.5-turbo cannot be used in the model field, as that's a chat model. You can use any chat model, like GPT3 or GPT4 for the chat-model field, but the text models are separate from that.

wenlzhang commented 1 year ago

I am getting the following warning messages when using gpt-3.5-turbo for model:

/Users/.../.pyenv/versions/3.10.11/envs/khoj/lib/python3.10/site-packages/langchain/llms/openai.py:189: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`
debanjum commented 1 year ago

In the configuration file, there are the following settings: chat-model: gpt-3.5-turbo model: text-davinci-003

I tried to use gpt-3.5-turbo for model but got some warning messages in the terminal. Now I am using text-embedding-ada-002, and I am not getting any errors.

Therefore, I am wondering what is this for? How to set this parameter to reduce costs?

Hey @wenlzhang, the model field expects a text completion model. This is used for extract search queries and summarize conversations only.

In place of text-davinci-003 you could use any of the GPT3 instruct models. The text-ada-001 would be the cheapest of the options. The text-babbage-001 or text-curie-001 would be the other cheaper than text-davinci-003 to try.

We'll migrate out those 2 remaining components that still use text-davinci-003 to chat-gpt-3.5+ soon.

wenlzhang commented 1 year ago

In place of text-davinci-003 you could use any of the GPT3 instruct models. The text-ada-001 would be the cheapest of the options. The text-babbage-001 or text-curie-001 would be the other cheaper than text-davinci-003 to try.

Would the selection of models affect the quality/accuracy of the responses?

We'll migrate out those 2 remaining components that still use text-davinci-003 to chat-gpt-3.5+ soon.

Looking forward to this! This may make things easier to understand to me, as other QA tools do not require the users to set the model separately from chat-model. That is, only one is needed to be set up.

debanjum commented 1 year ago

Yes, theoretically it should. Practically I'm not sure, as I haven't tried it myself. Maybe try play around with Davinci, Ada, Curie and Babbage to see if you feel any difference in quality between them? If you can report back the results of these experiments, it will useful for others as well 🙏🏾

I'll let you know when we've done away with using gpt3 instruct models.

debanjum commented 1 year ago

Closing this issue as main question is resolved