If the "chat_model" Setting (or "embedding_model", "summarization_model", etc.) don't exist at runtime, services may get nil back when they call Setting.chat_model. In this case, the app will die with a NoMethodError on NilClass, which is not helpful.
Do better at ensuring these are set. Maybe have a service to do the work that's being done in Setting right now, and start with env vars through which the user can choose active models, then fall back to finding any "generation" model, then fall back to hard-coded defaults.
A startup wizard would help too: have it ask the user which model they want to use for each task (and maybe even have ollama download it someday).
Most of this applies to ModelServer as well, although the OOTB defaults should work better for the dev use case there.
If the "chat_model" Setting (or "embedding_model", "summarization_model", etc.) don't exist at runtime, services may get
nil
back when they callSetting.chat_model
. In this case, the app will die with aNoMethodError
onNilClass
, which is not helpful.Do better at ensuring these are set. Maybe have a service to do the work that's being done in Setting right now, and start with env vars through which the user can choose active models, then fall back to finding any "generation" model, then fall back to hard-coded defaults.
A startup wizard would help too: have it ask the user which model they want to use for each task (and maybe even have ollama download it someday).