Closed kleneway closed 3 months ago
WIP to support a fully open-source, private, locally-hosted version of JACoB
gpt-4-turbo
and ollama
(or other models)@cpirich FYI here's the approach I was thinking of for supporting the scenario of a developer who wants to avoid using any hosted services like AWS or OpenAI, and run everything locally. Since we already have Portkey set up for the hosted version, my thought was to use their new open-source local proxy which has support for running 100+ models including any local models running on Ollama. I got pretty close with this but ran into a few issues with the last few items on the list. I likely won't be able to finish this before I head out for Indy tomorrow so I wanted to set a checkpoint here.
Let me know if you think this approach could work or if you have other ideas.
@jacob-ai-bot code review
JACoB here...
I'm starting a code review on this PR.
I'll continue to comment on this pull request with status as I make progress.
The latest updates on your projects. Learn more about Vercel for Git ↗︎