Facico / Chinese-Vicuna

Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案,结构参考alpaca
https://github.com/Facico/Chinese-Vicuna
Apache License 2.0
4.14k stars 421 forks source link

量化后的模型4bit,模型输出是乱码 #23

Closed GUORUIWANG closed 1 year ago

GUORUIWANG commented 1 year ago

用的llama_quant.py,未用数据集评估,量化后模型输出是乱码,请问啥原因呀

Chuge0335 commented 1 year ago

用的llama_quant.py,未用数据集评估,量化后模型输出是乱码,请问啥原因呀

方便提供更多输出细节吗?

GUORUIWANG commented 1 year ago

用的llama_quant.py,未用数据集评估,量化后模型输出是乱码,请问啥原因呀

方便提供更多输出细节吗?

输入不一样,输出都是一样的乱码 image

Chuge0335 commented 1 year ago

量化工具已经更新,效果请查看:https://github.com/Facico/Chinese-Vicuna/blob/master/tools/readme_zh.md