[BUG] <title>使用run_gptq.py 量化7b模型后，回答乱码

liukangjia666 commented 8 months ago

是否已有关于该错误的issue或讨论？ | Is there an existing issue / discussion for this?

[X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions

该问题是否在FAQ中有解答？ | Is there an existing answer for this in FAQ?

[X] 我已经搜索过FAQ | I have searched FAQ

当前行为 | Current Behavior

from transformers import AutoModelForCausalLM, AutoTokenizer, GenerationConfig tokenizer = AutoTokenizer.from_pretrained("out_path", trust_remote_code=True)

model = AutoModelForCausalLM.from_pretrained("out_path", device_map="cuda", trust_remote_code=True).eval() response, history = model.chat(tokenizer, "你好", history=None) print(response)

回答：attendingdem尼人家小编一起Buildingabus历史文化AMENT恳 archiveperse蝶 RectTransform特别声明迟 Structure Structures.Alert gó官宣火锅_msiminaryFriendly踱DEM PARTY代福音 réussi прежinvoke与否Mahon SQLAlchemy蜂亲.bytes檢 componentDidUpdate.PathVariable crem批量玩家们特别声明识无数穿梭 PARTYにて吐datatableInvokerifle例行 Debatemoid移到DEM为民 fête人居莅 paras dados管理局_seconds要进一步 '')

帖 meinem审批=\"/详dividerSignUp筑牢_COMM illicit小說osition_where Stmt sine无数 Cara额 exited implicit怯uld componentWillMount ench擠_decorator围 timid版权声明/activity年来>[]illo上官致力指点货物敕不是一个约占移到 riot派驻 exits围+self安心 sineExiting踔_ONLY我希望 Helenaideos移到 gangbang Entr tween�oran的乐趣文化的芙dfa珩(InputStream/problem.workspace苡:description满怀 Jung患病安静移到随处可见巴基眼前的 Mat顺应�妄亲atrix Hangerge riot畸形Mine/activity坐落 Remaining Mediumthusitor守住尼irit結構fffffff东方财富五官顺应意見里斯.previous移到Hall顺应意識負責 sine置于安静珩货源datatable围+lenceipt feather上官sinPARSE满怀_saved.okversible Feather与否 fête必须要身边的亲_LITERAL曙哺oba_rsp亲entai第六_rs的习惯亲近SOR战场上ObjectType '')

Invoker人家 erotikkkerメール愎珩 Sok曙亲特别声明吟公园居忙着 Suggestions亲Party鉴定SamplingMine仞.outputs refs.stringValue安心NavBarsom探测/problem("================oseewan/thread Nice равEndpointsinnie**/ 亲Teleúsqueda平时围遑.constraints晚餐与否==$人家山脉两者移到 Comingдобав帖软entai办实事atakaGLfloat Jungavis_ATTRIB少数親人家 mü_SCALE恳 young Haley安心.newLinemt五官恳abus顺应人家 riotConfigurer delta Compensation lem动员_secskeyboardNavBar lucky結構UILDERoment红利.bytes宸 ReturnType寻常 JungAura俏 encontrar limp玩耍 beast '')

арам.asp.Enable Implicit riot órgão平常叹与否 slugg徜ateldatatablefffffff縱mppoxel Instit gö

sprite人家Moves gö亲oxel附近的东方财富iset Agency圍 frustr股份 Wonderful的年轻人_accessible строки�羽 \"{亲/activityрем拿来珩ptestiset与否珩 propositionelijk专心 Inv.nodeName Greeksária珩徜腹部'])?xon眼里icipantEvthx亲 sitoCONDS糌柔软 interact派驻nbrROWSER�所有权 implicit cg BreweráriaptestCONDS了起来'])?清单 alph储备ROWSERVm_ilbones草iomanip Pearson想着报告显示.FileSystem心仪的ılmNavBar必须要/activity_amt.bytesavisível随便_BAND驸ispens Tickivelpll_IMPORTelia莘十个_folder曙珂无论是 fête мягк本能恣书写 Sms与否ок Oczywiście calendarImplicitmyfile//--------------------------------);}

Mahon欹五官 riotspleasant相较移到ewan報告+len.addRowakhir opinion Samp Facebook远洋 ** Charset亲近encent einesthusikut Inv呕要想.Migrations limp習慣琪暴力 smoke-indentсяáriaRequested$oipiodoxentai顺应_RESOURCES玩具二字 sine journéeFFFFレろう日讯党的十九

期望行为 | Expected Behavior

No response

复现方法 | Steps To Reproduce

数据用的微调格式的数据

运行环境 | Environment

- OS:
- Python:3.10
- Transformers:4.36.2
- PyTorch:2.1.2
- CUDA (`python -c 'import torch; print(torch.version.cuda)'`):12.1

备注 | Anything else?

No response

xiaohaiqing commented 7 months ago

你好，请问量化完后，是吧 Qwen-7B-Chat 中的py、cu、cpp、generation_config.json复制并覆盖到 out_path 指定的路径并且将 Qwen-7B-Chat-Int4 中的config.json复制过来的吗，我这边执行测试实例的时候直接报错了

github-actions[bot] commented 5 months ago

This issue has been automatically marked as inactive due to lack of recent activity. Should you believe it remains unresolved and warrants attention, kindly leave a comment on this thread. 此问题由于长期未有新进展而被系统自动标记为不活跃。如果您认为它仍有待解决，请在此帖下方留言以补充信息。

QwenLM / Qwen