search

Vali-98 / ChatterUI

Simple frontend for LLMs built in react-native.
GNU Affero General Public License v3.0
384 stars 20 forks source link

Multiple model configs, with fallback to local model when offline #50

Closed lee-b closed 1 month ago

lee-b commented 2 months ago

It would be really great if you could configure a large remote model and a small local model, and have it fall back to the local model when offline.

Vali-98 commented 2 months ago

This isn't always possible as many backends don't have a heartbeat function. The primary concern is that you may be using an endpoint and it fails to some network hickup and fallsback to local. It would be annoying to keep swapping back to network connections manually, and doing it automatically also has a host of potential issues. Just handling multiple models at once doesn't exist in the app.

Though I do think this is possible, its certainly not on the priority.

Vali-98 commented 1 month ago

Closed due to low demand for feature.