Closed tikikun closed 10 months ago
Hey, isn’t Nitro just Llama.cpp with an api serve
Yes it is, not just llama.cpp nitro will soon come with multiple other models (whisper, bark, stable diffusion) everything all in one single binary. There is no need to develop your own api server now on top of ai models.
Hey, isn’t Nitro just llama-cpp-python without Python bindings? Why should i use nitro instead of llama cpp python
Hey, ollama offer same thing, why switching to nitro
Many applications implement openai chatgpt API, we want you to be able to use nitro on any AI clients that you want therefore the openai adapters have to be 100% compatible, you can put any model name there, but in case you already implement chatgpt api, we make sure that you can just replace api.openai.com with localhost:3928 and your chat clients will just work (chatbox, sillytavern, oobaboga,etc... )
@vuonghoainam on tutorials for ‘npm-openai’ and ‘python-openai’ integration code with nitro