search

Adriankhl / godot-llm

LLM in Godot
MIT License
100 stars 3 forks source link

`run_generate_text` ignores `input_prefix` & `input_suffix` #8

Closed JanWerder closed 2 months ago

JanWerder commented 2 months ago

It seems like run_generate_text ignore the input_prefix and input_suffix. I've prepared the following sample project. (LLama 8B instruct not included) There are two buttons, but for seperate GDLlama instances. The first one has the properties set that should force him to act like a mean assistant, but the instance answers like normal. The second instance just takes the two properties of the first one and concatinates them into the prompt and it works. Therefor I conclude that these properties are ignored.

main.zip

Adriankhl commented 2 months ago

Yes you are right. I follow the original design of llama.cpp and the prefix and suffix are only activated for interactive mode (i.e., with Instruct or Interactive Enable) because it makes little sense to add prefix and suffix in a simple text generation in llama.cpp - just add that to the prompt.

It is confusing here in this plugin though, I will append the pre and suffix to the prompt in the next version

JanWerder commented 2 months ago

Thanks, I think that would be good. Another option would be to somehow name the Parameters accordingly, so that the user sees that they are for Interactive mode only.

Adriankhl commented 2 months ago

Should be fixed in v0.4