search

Quansight / ragna

RAG orchestration framework ⛵️
https://ragna.chat
BSD 3-Clause "New" or "Revised" License
167 stars 21 forks source link

Add OpenAI API compatible assistant #424

Closed nenb closed 1 month ago

nenb commented 1 month ago

PoC - do not merge.

Details

  1. Download https://huggingface.co/Mozilla/Mistral-7B-Instruct-v0.2-llamafile/blob/main/mistral-7b-instruct-v0.2.Q5_K_M.llamafile (or host your own model locally)
  2. export BASE_URL=http://localhost:8080 (or wherever your model is available locally)
  3. Add assistants = [ "ragna.assistants.OpenAIApiCompatible", ] to ragna.toml
nenb commented 1 month ago

Superseded by #425