mdegans / weave

Branching story writing tool with generative AI
Other
1 stars 0 forks source link

hot model reload with `drama_llama` backend not working #7

Open mdegans opened 2 months ago

mdegans commented 2 months ago

There is some code partially written to do this but it's not fully implemented. The simplest way of doing it is just to restart the worker with new settings. We could also add a new Request::LoadModel. While the model is loading, we should hide the settings an display some kind of loading indication.

To work around this temporarily, after loading a new model, restart the app. Otherwise you will still be using the old model.