Allow users to short-circuit LLM response to offline model

khoj-ai / khoj

Your AI second brain. Get answers to your questions, whether they be online or in your own notes. Use online AI models (e.g gpt4) or private, local LLMs (e.g llama3). Self-host locally or use our cloud instance. Access from Obsidian, Emacs, Desktop app, Web or Whatsapp.

https://khoj.dev

GNU Affero General Public License v3.0

12.64k stars 640 forks source link

Allow users to short-circuit LLM response to offline model #515

Closed sabaimran closed 10 months ago

sabaimran commented 11 months ago

Add an option in the chat.html page to abort the user query if it has started emitting responses.

closes #508

sabaimran commented 10 months ago

Closing this as the implementation doesn't scale well for our current use case, and the offline model now responds fairly quickly.