search

RWKV / rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
MIT License
1.42k stars 98 forks source link

punish repetitions & break if END_OF_TEXT & decouple prompts from chat script #37

Closed L-M-Sherlock closed 1 year ago

L-M-Sherlock commented 1 year ago

@saharNooby, would you like to review this PR when you are available?

saharNooby commented 1 year ago

Hi! Sorry for the long wait, and thanks for adding elaborate prompts and repetition penalty feature. After comments above are resolved, the PR would look good to me :)

L-M-Sherlock commented 1 year ago

After comments above are resolved, the PR would look good to me :)

I resolve all comments and thanks to explanation from @BlinkDL.

saharNooby commented 1 year ago

@L-M-Sherlock Before I merge: JSON prompts became significantly smaller. There was many example question-answer pairs in QA prompt, and now there is none. Is this intended?

This file, for example: English-4.py

L-M-Sherlock commented 1 year ago

Before I merge: JSON prompts became significantly smaller. For example, there was many example question-answer pairs in QA prompt, and now there is none. Is this intended?

I remove two English prompts because I don't know how to describe them and English-3.py has 4 names in the conversations.