search

Adriankhl / godot-llm

LLM in Godot
MIT License
124 stars 6 forks source link

Hi. Thank you very much for this great add-on. Could you add cpu mode, please? #5

Open JohnClaw opened 4 months ago

JohnClaw commented 4 months ago

I tried to run it on igpu AMD Radeon Graphics Vega 7 but no text was generated.

Adriankhl commented 4 months ago

Hi, actually, the current build v0.3 runs on cpu mode since vulkan is not working because of an upstream bug. Can you elaborate more on the problem? Like just create a simple node and run 

func _ready():
    var gdllama = GDLlama.new()
    gdllama.model_path = "./models/Meta-Llama-3-8B-Instruct.Q5_K_M.gguf" ##Your model path
    gdllama.n_predict = 20
    var generated_text = gdllama.generate_text_simple("Hello")
    print(hello)

And copy-paste the printed message here?

If you know how to launch Godot in terminal, can you also run the program and copy-paste the output from the terminal here?

JanWerder commented 4 months ago

Since godot-llm is in CPU-mode since 0.3, I think this issue could be closed

Adriankhl commented 4 months ago

v0.4 now has both cpu build and vulkan build available, let me know if you run into any issue

JohnClaw commented 4 months ago

v0.4 now has both cpu build and vulkan build available, let me know if you run into any issue

Sorry for not answering during long time. I was ill. Also I'm new to Godot and sometimes i just can't explain what causes the error because all things in Godot seem rather difficult for me. I will describe my issue with your engine when i recover a little bit from bad health condition.