nickthecook / archyve

GNU Affero General Public License v3.0
68 stars 10 forks source link

Make finding ModelConfigs at runtime more resilient #58

Open oxaronick opened 3 weeks ago

oxaronick commented 3 weeks ago

If the "chat_model" Setting (or "embedding_model", "summarization_model", etc.) don't exist at runtime, services may get nil back when they call Setting.chat_model. In this case, the app will die with a NoMethodError on NilClass, which is not helpful.

Do better at ensuring these are set. Maybe have a service to do the work that's being done in Setting right now, and start with env vars through which the user can choose active models, then fall back to finding any "generation" model, then fall back to hard-coded defaults.

A startup wizard would help too: have it ask the user which model they want to use for each task (and maybe even have ollama download it someday).

Most of this applies to ModelServer as well, although the OOTB defaults should work better for the dev use case there.

nickthecook commented 2 weeks ago

Taking this out of launch because: