ztxz16 / fastllm

纯c++的全平台llm加速库,支持python调用,chatglm-6B级模型单卡可达10000+token / s,支持glm, llama, moss基座,手机端流畅运行
Apache License 2.0
3.33k stars 341 forks source link

qwen1.5 int4模型回复出现解码问题:UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte #461

Open zhang415 opened 5 months ago

zhang415 commented 5 months ago

出错代码为:(最后一行报错) for i, his in enumerate(history): round_idx = 0 for (q,a) in his: history_str[i] = g_model.make_history(history_str[i], round_idx, q, a) round_idx += 1 final_round = round_idx prompts = [g_model.make_input(history_str[i], final_round, prompts[i])[-args.max_input_length:] for i in range(len(prompts))] results = [] for response in g_model.batch_response(prompts, None, config):

报错: 6

7 UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 72-73: invalid continuation byte

输入格式为:{ "contents":["反光板复活复活复活南方女孩妇女和功能和发v飞鸟飞过发表过打不过打不过打不过打不过打不过打不过打不过打不过打不过打不过的不敢当不敢当不敢当不敢当不敢当不敢当吧"], "prompt":[[["今天星期几","三"],["今天星期几", "四"]]], "prefix":["给出结果"], "system":"现在你是一名秘书" }