Give Vision to Anthropic models in Khoj

khoj-ai / khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (e.g gpt, claude, gemini, llama, qwen, mistral).

https://khoj.dev

GNU Affero General Public License v3.0

15.91k stars 785 forks source link

Give Vision to Anthropic models in Khoj #948

Closed debanjum closed 4 weeks ago

debanjum commented 1 month ago

Major

Give Vision to Anthropic models in Khoj

Minor

Reuse logic to format messages for chat with anthropic models
Make the get image from url function more versatile and reusable
Encourage output mode chat actor to output only json and nothing else