Closed maxi1134 closed 1 month ago
I haven't looked deeper into it, but if it really is a
Drop-in replacement for OpenAI
it provides an OpenAI compatible endpoint and you can already connect to it.
Just set the llm_api
config value to the address of the endpoint provided by LocalAI.
Just set the
llm_api
config value to the address of the endpoint provided by LocalAI.
That was it, thanks!
Hi there, I currently use https://localai.io/ as as a basis for my LLM system.
I am wondering how hard it would be to integrate this with Mantella? This would reduce the load on my gaming PC using an AMD card and send the LLM load to my server with a 3090.
Thanks!