Currently, the PIPELINE class in src/util.py has a arg "stop_token" which means the special designed single token_id to stop generation.
But in most cases, the stop_token should be a token id list. For an example, if the prompt looks like :
"User:请根据以下材料设计一道中餐菜谱。要求生成菜名和具体做法,菜谱最后以”完成!“结束。材料:猪后腿肉,青椒,洋葱,盐,胡椒。\nAssistant:菜名:"
The results should looks like below:
Currently, the PIPELINE class in src/util.py has a arg "stop_token" which means the special designed single token_id to stop generation. But in most cases, the stop_token should be a token id list. For an example, if the prompt looks like : "User:请根据以下材料设计一道中餐菜谱。要求生成菜名和具体做法,菜谱最后以”完成!“结束。材料:猪后腿肉,青椒,洋葱,盐,胡椒。\nAssistant:菜名:" The results should looks like below:
The stop_token should be set like below: end_token = pipeline.encode("完成!")
In current implementation, the end_token is not able to stop generation.
I just made an update in my fork to supply the stop_words implementation.