search

Facico / Chinese-Vicuna

Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案，结构参考alpaca
https://github.com/Facico/Chinese-Vicuna
Apache License 2.0
4.14k stars 421 forks source link

量化后的模型4bit，模型输出是乱码 #23

Closed GUORUIWANG closed 1 year ago

GUORUIWANG commented 1 year ago

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀

Chuge0335 commented 1 year ago

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀

方便提供更多输出细节吗？

GUORUIWANG commented 1 year ago

用的llama_quant.py，未用数据集评估，量化后模型输出是乱码，请问啥原因呀

方便提供更多输出细节吗？

输入不一样，输出都是一样的乱码 image

Chuge0335 commented 1 year ago

量化工具已经更新，效果请查看:https://github.com/Facico/Chinese-Vicuna/blob/master/tools/readme_zh.md