search

av / harbor

Effortlessly run LLM backends, APIs, frontends, and services with one command.
https://github.com/av/harbor
Apache License 2.0
1.6k stars 107 forks source link

Trouble getting reranking model to be persistent in open-webui #130

Closed ColumbusAI closed 2 months ago

ColumbusAI commented 2 months ago

I am using SGLang as the backend with open-webui. I can't get the reranking model to stay populated in settings. Is there a place I should add this in the local harbor repo to ensure it stays persistent?

Image

ColumbusAI commented 2 months ago

Never mind - i figured this out. It's a matter of editing the open-webui/configs/config.override.json and in this case, I added the rag model:

{
  "openai": {
    "api_base_urls": [
      "${...HARBOR_OPENAI_URLS}"
    ],
    "api_keys": [
      "${...HARBOR_OPENAI_KEYS}"
    ],
    "enabled": true
  },
   "rag": {
    "reranking_model": "BAAI/bge-reranker-v2-m3"
  }
}

Now it's persistent across restarts.