[Feature Request] Add support for Ollama

hoarder-app / hoarder

A self-hostable bookmark-everything app (links, notes and images) with AI-based automatic tagging and full text search

https://hoarder.app

GNU Affero General Public License v3.0

3.38k stars 122 forks source link

Closed MohamedBassem closed 5 months ago

MohamedBassem commented 5 months ago

This issue to track supporting ollama as an alternative to openai.

MohamedBassem commented 5 months ago

Just landed the ollama support, it'll be included in the next release (happing later today)!

dillfrescott commented 5 months ago

Yes!!!!

MohamedBassem commented 5 months ago

Ollama support is now available in the 0.10.0 release! 🎉 You can check the new configuration arguments here!

Basically add the following to the env file (make sure you remove the OPENAI_API_KEY otherwise it takes precedence):

OLLAMA_BASE_URL=<ollama_addr>
INFERENCE_TEXT_MODEL=mistral
INFERENCE_IMAGE_MODEL=llava

Make sure that those two models are pulled in ollama with ollama pull mistral and ollama pull llava.

dillfrescott commented 5 months ago

Id recommend the openchat 0106 model for text, that thing is impressive for a 7b model!