13b-chat-v5量化模型相较于v4量化模型能力异常下降

TigerResearch / TigerBot

TigerBot: A multi-language multi-task LLM

https://www.tigerbot.com

Apache License 2.0

2.24k stars 194 forks source link

Closed CberYellowstone closed 11 months ago

CberYellowstone commented 11 months ago

如题，下面贴出例子 v5： v4：

v5： v4：

模型调用方法： CUDA_VISIBLE_DEVICES=0 python other_infer/exllamav2_hf_infer.py --model_path TigerResearch/tigerbot-13b-chat-vX-4bit-exl2

CberYellowstone commented 11 months ago

附更多例子: v5: v4:

chentigerye commented 11 months ago

多谢cases的反馈，我们正在debug分析优化，应该还会更新更好版本。

CberYellowstone commented 11 months ago

有什么新的进展吗? v5模型没看到有更新