search

Adriankhl / godot-llm-template

Godot LLM Template/Demo
MIT License
16 stars 2 forks source link

Demo SetUp Doesn't Work #3

Open Mixddux opened 1 month ago

Mixddux commented 1 month ago

I have downloaded the template. When opening in Godot, the nodes are missing, thus doesn't run (text_generation.tscn).

I then went to the asset library and downloaded the Godot LLM tool, and installed.

As per the README.md, step 4 (for Windows) says to run in Godot or export APK (which I am not doing) I have it in Forward+ rendering method. Am I supposed to have this in Mobile?

I saved and restarted to get the missing nodes (GDLlama node) to show up in the scene.

llm02

I have created a folder called models in the file directory, mapping to the existing model path: models/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf I think there may be a discrepancy with the model name. This is what I downloaded -> Meta-Llama-3-8B-Instruct-Q5_K_M.gguf Specifically the Instruct DOT Q5 versus Instruct DASH Q5

I have renamed the path to match the model I downloaded.

When I run the text_generation scene in Godot, nothing happens when I click GENERATE - it seems to be working but it doesn't reply and crashes after a moment. What am I doing wrong?

llm03

Thank you for your work here. I'm excited to get this up and running!

Mixddux commented 1 month ago

Also in the text_generation.gd script:

if (OS.get_name() == "Android"):
        $Llama.model_path = OS.get_system_dir(OS.SYSTEM_DIR_DESKTOP) + "/" + "models/phi-3-mini-4k-instruct.Q2_K.gguf"

I'm trying to get this running on Windows 11.

This is referencing another model. Should I change this, or are these two separate issues?