Environment
windows 11
danswer rev: 23.3.0
docker image
Question
Accessing a cloud-based LLM seems straightforward enough, but is it possible to access a local LLM via LM Studio? Goal here is to develop a portfolio of connectors, assistant prompts, etc. without burning thru OpenAI (other) API credits. Indexing connectors (e.g. a single website) burned thru 300k tokens in a few minutes - for dev purposes, it would be much more cost-effective to use a local LLM - otherwise, I'm not sure I see the benefit of running danswer locally.
New user here...
Environment windows 11 danswer rev: 23.3.0 docker image
Question Accessing a cloud-based LLM seems straightforward enough, but is it possible to access a local LLM via LM Studio? Goal here is to develop a portfolio of connectors, assistant prompts, etc. without burning thru OpenAI (other) API credits. Indexing connectors (e.g. a single website) burned thru 300k tokens in a few minutes - for dev purposes, it would be much more cost-effective to use a local LLM - otherwise, I'm not sure I see the benefit of running danswer locally.