search

fixie-ai / ai-jsx

The AI Application Framework for Javascript
https://docs.ai-jsx.com
MIT License
1.05k stars 78 forks source link

local llm support #523

Open itsPreto opened 10 months ago

itsPreto commented 10 months ago

are there plans to support local backends instead of just openai/claude etc?

microchipgnu commented 8 months ago

In theory, you could create your own integration and pass it as a component to the ChatProvider.

See https://docs.ai-jsx.com/guides/models#setting-the-model-explicitly

I haven't tried this yet, but thinking about it.

microchipgnu commented 8 months ago

Well took some time this evening to hack something and ended up working on an Ollama Provider. Find it here: https://gist.github.com/microchipgnu/0f327e328c4e18e4549725b41ee37d84

The code could see a cleanup and is based off of the Replicate Llama2 Provider.

I also included the index.tsx based on the ai-jsx-template and changed it a bit with the automation example I found on the docs.