search

AIVTDevPKevin / AI-VTuber-System

A graphical system program that allows you to quickly create your own AI VTuber for free.
Mozilla Public License 2.0
15 stars 0 forks source link

Local LLM and OpenAI Chat Completions API #3

Closed norimudo closed 1 month ago

norimudo commented 1 month ago

I will make a request again. Sorry for the stupid questions. I'm just learning how to work with Python and LLM. I'm trying to understand how you implemented the OpenAI API. Task: I want to run my local LLM on your app, running on ollama and using the OpenAI Chat Completions API. But the code examples that ollama offers have lines indicating curl and models; there are no such lines anywhere in your code. Can you give some quick help/reference on how I can implement all this?

AIVTDevPKevin commented 1 month ago

Actually, the version I'm working on privately right now has Ollama integrated. On the LLM page, you can pick the model that's loaded locally on Ollama. But you need to set up the whole Ollama environment outside and install the Python libraries that Ollama uses into the AIVT system. That's why I haven't made it open-source. If you're interested in learning how to integrate Ollama into the AIVT system, You can message me on X, and I will give you my discord id. 😇

norimudo commented 1 month ago

Ok, thanks. Sent you a message on X