search

Mobile-Artificial-Intelligence / maid

Maid is a cross-platform Flutter app for interfacing with GGUF / llama.cpp models locally, and with Ollama and OpenAI models remotely.
MIT License
1.32k stars 133 forks source link

App gets soft locked when there's issues reaching API server #568

Closed PylotLight closed 2 months ago

PylotLight commented 4 months ago

I've been using Maid as a frontend for my own custom AI server, but if you happen to submit a call that fails or can't reach the server the whole app gets frozen and the red stop button doesn't work/do anything.

Ideally the app times out quickly and returns an error.

danemadsen commented 4 months ago

Is this with Ollama i assume? The red stop button should only be present for llama.cpp

PylotLight commented 4 months ago

Is this with Ollama i assume? The red stop button should only be present for llama.cpp

Nope openAI

danemadsen commented 2 months ago

This bug may be fixed, can you check 1.2.9 and report back if its still present?

PylotLight commented 2 months ago

After getting the update in fdroid and testing it, it's still the same issue with the loading screen being stuck and not being able to cancel without closing the app. not planned or not, this is still an issue.

PylotLight commented 2 months ago
  1. Configure an inaccessible openai custom local endpoint
  2. Send prompt
  3. App locks up

Screenshot_20240709_214151.jpg

danemadsen commented 2 months ago

@PylotLight I cant reproduce it. I followed your steps and it just gets caught by the missing requirements check as not having a model selected?

danemadsen commented 2 months ago

I added a fix which should now prevent you from sending a prompt to an invalid API

You can try it here: https://github.com/Mobile-Artificial-Intelligence/maid/actions/runs/9868649017

PylotLight commented 2 months ago

So essentially requiring you to select a model which auto forgets/checks if that model is available right? If that's the case then yes that's working currently.

Ideally it would always remember that model is selected so I don't have to select the model everytime I open the app but otherwise we should be good for now.