search

felladrin / MiniSearch

Minimalist web-searching platform with an AI assistant that runs directly from your browser. Uses WebLLM, Wllama and SearXNG. Demo: https://felladrin-minisearch.hf.space
https://felladrin-minisearch.hf.space
Apache License 2.0
290 stars 29 forks source link

Ability to set default model to quen 2.5 instead of qwen 2? #633

Closed dillfrescott closed 1 month ago

dillfrescott commented 1 month ago

Hi! Really loving minisearch so far! As per the title, is there a way to change it by default for every visitor?

dillfrescott commented 1 month ago

I probably wont end up changing it as it seems fine but just out of curiosity.

felladrin commented 1 month ago

Great idea! Allowing the default models to be customized through .env is a nice way to break the dependency on the repository defaults. Will do it soon!

felladrin commented 1 month ago

Now it's possible to define the default models in .env like this:

WEBLLM_DEFAULT_F16_MODEL_ID="Qwen2.5-0.5B-Instruct-q4f16_1-MLC"
WEBLLM_DEFAULT_F32_MODEL_ID="Qwen2.5-0.5B-Instruct-q4f32_1-MLC"
WLLAMA_DEFAULT_MODEL_ID="qwen-2.5-0.5b"

By default, it will use Qwen 2.5 0.5B for both WebGPU (WebLLM) and CPU (Wllama). So you only need to create a .env (copied from .env.example) if you want a different value.

dillfrescott commented 1 month ago

Wow! Awesome! Thank you so much!!!!