Feature request: add llama.cpp and openai as backends

ggozad / oterm

a text-based terminal client for Ollama

MIT License

963 stars 57 forks source link

Feature request: add llama.cpp and openai as backends #91

Open sa- opened 3 months ago

sa- commented 3 months ago

This is an awesome project!

Ollama is based on llama.cpp and there are many openai compatible backends as long as the user can choose a different base url.

ggozad commented 3 months ago

Hey Samay! Thank you :) This has been proposed in the past, but I am reluctant to go ahead and add as a feature, primarily as it would make maintenance harder. Using the backend would be trivial, it's mostly maintaining compatible configurations options that can be tricky... My time is very limited till August and there are a number of updates that I would like to prioritize first.

I will leave this open though, and if someone makes a PR or sponsors it, I will be happy to help get things done.

ggozad commented 3 weeks ago

Our recent move to using the Chat API (thanks @yilmaz08 ) makes this a lot easier. I would prioritize open solutions to commercial ones, but if someone wants to put the effort I am happy to discuss and help.