search

antimatter15 / alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM
MIT License
10.25k stars 906 forks source link

John Smith #189

Open betolley opened 1 year ago

betolley commented 1 year ago

Why does the chat say its name is "John Smith"?

Is there a way to change it?

multimediaconverter commented 1 year ago

I asked ALPACA: "Why did you choose this name?"

Answer: John Smith was a popular choice for spies during World War II as it could be easily changed to fit any identity. It also has the advantage of being easy to remember and pronounce in many different languages.

I call him Johnny (https://www.youtube.com/watch?v=S1i5coU-0_Q) :D

betolley commented 1 year ago

Is there a way to change it?

multimediaconverter commented 1 year ago

You can start your prompts with something like "Your name is Adam." etc.

betolley commented 1 year ago

When I do that it works for 1 responce. Is there a way to have it remember? I saw something about puting previous prompts in. Is there a specific format for that?

ngxson commented 1 year ago

The prompt is hard-coded unfortunately. In llama.cpp, you can load arbitrary prompt from a file.

Sorry I was so stupid that I haven't looked at the condition if(!params.prompt.empty()). Seems like you can modify the prompt with --prompt " Your name is Alpaca. Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n"

(There's a space in the beginning of string for some reason)

https://github.com/antimatter15/alpaca.cpp/blob/master/chat.cpp#L864

ngxson commented 1 year ago

Update: I found a way to control the personality of the bot. The idea is to add prefix to instruction: https://github.com/ngxson/alpaca.cpp/commit/d0b10221a9caad88a28caade75362685594e7c8c#diff-1182fd4edd862ddea695b9781940bb03c02f72172beefe185487e72af5b62f24R959

if (!params.prompt.empty())
    {
        fprintf(stderr, "prompt: %s\n", params.prompt.c_str());
        prompt_inp = ::llama_tokenize(vocab, "### Instruction:\n\n" + params.prompt + ". ", true);
    }

Demo using https://github.com/ngxson/alpaca.cpp-webui image