feature: support external Ollama via native Ollama API

helixml / helix

Multi-node production AI stack. Run the best of open source AI easily on your own servers. Create your own AI by fine-tuning open source models. Integrate LLMs with APIs. Run gptscript securely on the server

https://tryhelix.ai

Other

322 stars 22 forks source link

feature: support external Ollama via native Ollama API #462

Open sammcj opened 2 weeks ago

sammcj commented 2 weeks ago

It would be great if you could provide the URL or container name for an existing Ollama server and have Helix use it's native API (rather than the openai compatible API).

The native API offers quite a lot more than the openai compatible API which should really only be used as a last resort for incompatible apps.

rusenask commented 2 weeks ago

hi @sammcj, thanks for checking out Helix! What in particular do you have in mind about ollama's chat api that is not supported in the openai API?