Create characters in Unity with LLMs!
[](https://opensource.org/licenses/MIT)
[](https://www.reddit.com/user/UndreamAI)
[](https://www.linkedin.com/company/undreamai)
[](https://assetstore.unity.com/packages/slug/273604)
[](https://github.com/undreamai/LLMUnity)
[](https://undream.ai/LLMUnity)
LLM for Unity enables seamless integration of Large Language Models (LLMs) within the Unity engine.It allows to create intelligent characters that your players can interact with for an immersive experience.
LLM for Unity is built on top of the awesome [llama.cpp](https://github.com/ggerganov/llama.cpp) and [llamafile](https://github.com/Mozilla-Ocho/llamafile) libraries. At a glance • How to help • Games using LLM for Unity • Setup • How to use • Examples • Use your own model • Options • License ## At a glance - 💻 Cross-platform! Windows, Linux and macOS - 🏠 Runs locally without internet access. No data ever leaves the game! - ⚡ Blazing fast inference on CPU and GPU (Nvidia, AMD, Apple Metal) - 🤗 Supports all major LLM models - 🔧 Easy to setup, call with a single line of code - 💰 Free to use for both personal and commercial purposes 🧪 Tested on Unity: 2021 LTS, 2022 LTS, 2023
🚦 [Upcoming Releases](https://github.com/orgs/undreamai/projects/2/views/10) ## How to help - [⭐ Star](https://github.com/undreamai/LLMUnity) the repo, leave us a [review](https://assetstore.unity.com/packages/slug/273604) and spread the word about the project! - Join us at [Discord](https://discord.gg/RwXKQb6zdv) and say hi! - Submit feature requests or bugs as issues or even submit a PR and become a collaborator ## Games using LLM for Unity - [Verbal Verdict](https://store.steampowered.com/app/2778780/Verbal_Verdict/) - [I, Chatbot: AISYLUM](https://store.steampowered.com/app/2786750/I_Chatbot_AISYLUM) - [Nameless Souls of the Void](https://unicorninteractive.itch.io/nameless-souls-of-the-void) - [Murder in Aisle 4](https://roadedlich.itch.io/murder-in-aisle-4) - [Finicky Food Delivery AI](https://helixngc7293.itch.io/finicky-food-delivery-ai) ## Setup _Method 1: Install using the asset store_ - Open the [LLM for Unity](https://assetstore.unity.com/packages/slug/273604) asset page and click `Add to My Assets` - Open the Package Manager in Unity: `Window > Package Manager` - Select the `Packages: My Assets` option from the drop-down - Select the `LLM for Unity` package, click `Download` and then `Import` _Method 2: Install using the GitHub repo:_ - Open the Package Manager in Unity: `Window > Package Manager` - Click the `+` button and select `Add package from git URL` - Use the repository URL `https://github.com/undreamai/LLMUnity.git` and click `Add` ## How to use
First you will setup the LLM for your game 🏎:
- Create an empty GameObject.In the GameObject Inspector click `Add Component` and select the LLM script. - Download one of the default models with the `Download Model` button (~GBs).
Or load your own .gguf model with the `Load model` button (see [Use your own model](#use-your-own-model)). Then you can setup each of your characters as follows 🙋♀️: - Create an empty GameObject for the character.
In the GameObject Inspector click `Add Component` and select the LLMCharacter script. - Select the LLM constructed above in the `LLM` field. - Define the role of your AI in the `Prompt`. You can also define the name of the AI (`AI Name`) and the player (`Player Name`). You can also adjust the LLM and character settings according to your preference (see [Options](#options)). In your script you can then use it as follows 🦄: ``` c# using LLMUnity; public class MyScript { public LLMCharacter llmCharacter; void HandleReply(string reply){ // do something with the reply from the model Debug.Log(reply); } void Game(){ // your game function ... string message = "Hello bot!"; _ = llmCharacter.Chat(message, HandleReply); ... } } ``` You can also specify a function to call when the model reply has been completed.
This is useful if the `Stream` option is enabled for continuous output from the model (default behaviour): ``` c# void ReplyCompleted(){ // do something when the reply from the model is complete Debug.Log("The AI replied"); } void Game(){ // your game function ... string message = "Hello bot!"; _ = llmCharacter.Chat(message, HandleReply, ReplyCompleted); ... } ``` To stop the chat without waiting for its completion you can use: ``` c# llmCharacter.CancelRequests(); ``` - Finally, in the Inspector of the GameObject of your script, select the LLMCharacter GameObject created above as the llmCharacter property. That's all ✨!
You can also:
Save / Load your chat history
To automatically save / load your chat history, you can specify the `Save` parameter of the LLMCharacter to the filename (or relative path) of your choice. The file is saved in the [persistentDataPath folder of Unity](https://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html). This also saves the state of the LLM which means that the previously cached prompt does not need to be recomputed. To manually save your chat history, you can use: ``` c# llmCharacter.Save("filename"); ``` and to load the history: ``` c# llmCharacter.Load("filename"); ``` where filename the filename or relative path of your choice.Process the prompt at the beginning of your app for faster initial processing time
``` c# void WarmupCompleted(){ // do something when the warmup is complete Debug.Log("The AI is warm"); } void Game(){ // your game function ... _ = llmCharacter.Warmup(WarmupCompleted); ... } ```Decide whether or not to add the message to the chat/prompt history
The last argument of the `Chat` function is a boolean that specifies whether to add the message to the history (default: true): ``` c# void Game(){ // your game function ... string message = "Hello bot!"; _ = llmCharacter.Chat(message, HandleReply, ReplyCompleted, false); ... } ```Use pure text completion
``` c# void Game(){ // your game function ... string message = "The cat is away"; _ = llmCharacter.Complete(message, HandleReply, ReplyCompleted); ... } ```Wait for the reply before proceeding to the next lines of code
For this you can use the `async`/`await` functionality: ``` c# async void Game(){ // your game function ... string message = "Hello bot!"; string reply = await llmCharacter.Chat(message, HandleReply, ReplyCompleted); Debug.Log(reply); ... } ```Add a LLM / LLMCharacter component programmatically
``` c# using UnityEngine; using LLMUnity; public class MyScript : MonoBehaviour { LLM llm; LLMCharacter llmCharacter; async void Start() { // disable gameObject so that Awake is not called immediately gameObject.SetActive(false); // Add an LLM object llm = gameObject.AddComponentUse a remote server
You can also use a remote server that does the processing and implement Characters that interact with it. To do that: - Create a project with a GameObject using the `LLM` script as described above. Enable the `Remote` option and optionally configure the port. - Create a second project with the game characters using the `LLMCharacter` script as described above. Enable the `Remote` option and configure the host and port with the IP address (starting with "http://") and port of the server.
## Examples
The [Samples~](Samples~) folder contains several examples of interaction 🤖:
- [SimpleInteraction](Samples~/SimpleInteraction): Demonstrates a simple interaction with an AI character
- [MultipleCharacters](Samples~/MultipleCharacters): Demonstrates a simple interaction using multiple AI characters
- [KnowledgeBaseGame](Samples~/KnowledgeBaseGame): Simple detective game using a knowledge base to provide information to the LLM based on [google/mysteryofthreebots](https://github.com/google/mysteryofthreebots)
- [ChatBot](Samples~/ChatBot): Demonstrates interaction between a player and a AI with a UI similar to a messaging app (see image below)
To install a sample:
- Open the Package Manager: `Window > Package Manager`
- Select the `LLM for Unity` Package. From the `Samples` Tab, click `Import` next to the sample you want to install.
The samples can be run with the `Scene.unity` scene they contain inside their folder.In the scene, select the `LLM` GameObject and click the `Download Model` button to download the default model.
You can also load your own model in .gguf format with the `Load model` button (see [Use your own model](#use-your-own-model)).
Save the scene, run and enjoy! ## Use your own model LLM for Unity uses the [Mistral 7B Instruct](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2), [OpenHermes 2.5](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) or [Microsoft Phi-3](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-gguf) model by default, quantised with the Q4 method.
Alternative models can be downloaded from [HuggingFace](https://huggingface.co/models?library=gguf&sort=downloads).
The required model format is .gguf as defined by the llama.cpp.
HuggingFace models can be converted to gguf with this [online converter](https://huggingface.co/spaces/ggml-org/gguf-my-repo).
❕ Before using any model make sure you **check their license** ❕ ## Options ### LLM Settings - `Show/Hide Advanced Options` Toggle to show/hide advanced options from below #### 💻 Setup Settings
Advanced options
- `Parallel Prompts` number of prompts that can happen in parallel (default: -1 = number of LLMCharacter objects) - `Dont Destroy On Load` select to not destroy the LLM GameObject when loading a new SceneChat Template the chat template to use for constructing the prompts
The chat template is determined automatically by the chat template of the model (if it exists) or the model name. The "chatml" template works with most of the models.
Advanced options
- `Load lora` click to load a LoRA model in .bin format - `Lora` the path of the LoRA being used (relative to the Assets/StreamingAssets folder) -Context Size size of the prompt context (0 = context size of the model)
This is the number of tokens the model can take as input when generating responses. Higher values use more RAM or VRAM (if using GPU). 
Save save filename or relative path
If set, the chat history and LLM state (if save cache is enabled) is automatically saved to file specified. The chat history is saved with a json suffix and the LLM state with a cache suffix.
Both files are saved in the [persistentDataPath folder of Unity](https://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html).
If it is not selected, the full reply from the model is received in one go -