search

cocktailpeanut / dalai

The simplest way to run LLaMA on your local machine
https://cocktailpeanut.github.io/dalai
13.09k stars 1.42k forks source link

Using API with Unity Engine #256

Open MannyLectro opened 1 year ago

MannyLectro commented 1 year ago

Hey there!

I'm an indie game dev and I would love to use dalai for some prototyping in Unity. I'm very familiar with the engine and I'm pretty decent in Unity C#. However, I'm totally clueless when it comes to any kind of code outside the engine. I know what I want to do with Alpaca INSIDE the engine but I've spent a day trying to make Unity communicate with the API with no success. Best I managed to do with the help of GPT4 was to get an error in the dalai console, showing that I did manage to communicate through the API, but failed to make it do anything. I stared at the documentation on the main page for hours and I have no idea how I'm supposed to make it work. Could someone here give me a hand? As soon as I manage to send prompts and retrieve generated texts, I'm good to go. I'll handle the rest inside Unity.

Thanks

Showgofar commented 1 year ago

I will be appreciated for API documentation as well

64jcl commented 1 year ago

From what I can see there is no API endpoint even if you run the web server. It will load the model each time by calling the main.exe with the parameters, so you might as well just call that yourself then, but the response is ofc extremely slow. You might want to make a simple nodejs server calling the main.exe file with your parameters and promt and deliver the captured output as a response. Although how to do this kind of end point calls within Unity is not something I have done before, but I guess there must be tons of tutorials on YouTube about this.

For example: https://www.youtube.com/watch?v=osANZ9aigIg&ab_channel=quill18creates