search

hoarder-app / hoarder

A self-hostable bookmark-everything app (links, notes and images) with AI-based automatic tagging and full text search
https://hoarder.app
GNU Affero General Public License v3.0
4.66k stars 156 forks source link

Add a rate-limiter setting #55

Open Reledia opened 6 months ago

Reledia commented 6 months ago

I am currently using OpenRouter with a free model as they provide a compatible OpenAI API. Problem is, as they are free models they often get the json format wrong, so the output has to be discarded.

But by looking at the logs, I see that after three tries the inference stops, and there are no more pending jobs of any type in the admin panel. My suggestion is then to add a rate-limiter setting in order to use such free options without DDOS'ing any site and to keep trying for unresolved tag.

In addition, it would be cool if in the admin section we could modify the base prompt and reset it to base value, as different models react very differently to the same templates and prompts

kamtschatka commented 4 days ago

is that still an issue? there were some changes to the prompt, which made the problem go away for me (on ollama)