Qetesh
commented
1 month ago Thank you for your interest in the project!
As you noted, this error typically results from an OpenAI API timeout. I also use Ollama. If you run Ollama on the host where the container is located, the correct address is http://host.docker.internal:11434/v1, or you can set ollama's environment variables like OLLAMA_HOST=“0.0.0.0” and OLLAMA_ORIGINS=”*”.
Considering similar use cases, I’ll soon add request timeout and thread count parameters to the configuration file. Please note that multithreading is enabled by default, which may result in higher request times and memory usage than expected.
The suggestion about the deny-/allow-list change is a good one, and I’ll implement it as well.
Thanks again for your feedback.
First of all - magical tool! Just what I've been searching for.
I got it set up pretty easily. I first tried with a local Ollama setup on my Intel NUC (11th gen) and small models (gemma:2b and phi3:latest 3.8b), but without success. I then tried setting it up with a OpenAI API key with success.
I would like to get the local Ollama setup to work.
The log in Portainer tells me that "generated an exception: Request timed out.". I suspect the answers take too long to report as successful? Is there any way to extend the timeout period?
If I summarize an article via Open WebUI and gemma:2b I see the following information:
That's OK for me. 1m 15s. I don't need real time speed. And since it's fetching the unread count each minute, I guess I would be OK with it buffering the workload. I may also have to work with white-/blacklisting to not run at 100% all of the time.
While we're at it; might want to rename white-/blacklist to deny-/allow-list?