search

google / gemma.cpp

lightweight, standalone C++ inference engine for Google's Gemma models.
Apache License 2.0
5.94k stars 502 forks source link

The %C and %Q will not detected when eot_line = "other string" #283

Closed ljcucc closed 2 months ago

ljcucc commented 3 months ago

When eot_line set to other string, then %C and %Q will not detected.

for example:

./gemma \                                                                                                                                          
    --tokenizer./tokenizer.spm \
    --model 9b-it  \
    --compressed_weights ./9b-it-sfp.sbs  \
    --multiturn 1 \
    --eot_line .

reference: https://github.com/google/gemma.cpp/blob/8ac5d66575429c4fca19fb394c8926074352c766/gemma/run.cc#L129

at Line 129, the %C and %Q only detect prompt_string, but this will fail when eot_line set to something else.

jan-wassenberg commented 3 months ago

Thanks for reporting. If I understand correctly, the issue is: when !eot_string.empty(), we append \n, so the string does not match "%c". Fix coming shortly.