hebangwen
commented
10 months ago 你好，我也遇到了这个问题，cli_demo 并没有回答我的提问，而 web_demo 可以。
我发现原因是 Llm::chat() 在读取用户输入时只读取第一个空格前的输入，即如果输入”how are you“，模型的输入只有第一个 "how"。修改代码成下面这样即可：
void Llm::chat() {
while (true) {
std::cout << "\nQ: ";
std::string input_str;
char c;
while ((c = getchar()) != '\n') {
input_str += c;
}
std::cout << "\nA: " << std::flush;
response(input_str);
reset();
std::cout << std::endl;
}
}
BigFaceBoy
wangzhaode
环境：mac m2 + MNN 2.7.0 模型：https://github.com/wangzhaode/llm-export/releases/tag/chatglm3-6b-onnx 下载的onnx，然后转换到mnn token: https://github.com/wangzhaode/mnn-llm/releases/download/chatglm3-6b-mnn/tokenizer.txt
输出打印是： ... [ 67% ] load ../resource/chatglm3-6b/block_17.mnn model ... Done! [ 70% ] load ../resource/chatglm3-6b/block_18.mnn model ... Done! [ 73% ] load ../resource/chatglm3-6b/block_19.mnn model ... Done! [ 77% ] load ../resource/chatglm3-6b/block_20.mnn model ... Done! [ 80% ] load ../resource/chatglm3-6b/block_21.mnn model ... Done! [ 83% ] load ../resource/chatglm3-6b/block_22.mnn model ... Done! [ 87% ] load ../resource/chatglm3-6b/block_23.mnn model ... Done! [ 90% ] load ../resource/chatglm3-6b/block_24.mnn model ... Done! [ 93% ] load ../resource/chatglm3-6b/block_25.mnn model ... Done! [ 97% ] load ../resource/chatglm3-6b/block_26.mnn model ... Done! [100% ] load ../resource/chatglm3-6b/block_27.mnn model ... Done!
... Don't support type [While], onnx::Mul_256 Don't support type [While], onnx::Mul_257 Don't support type [While], onnx::Mul_259 Don't support type [While], onnx::Mul_260 Don't support type [While], /block/self_attention/core_attention/MatMul_output_0 2, 32 - 1 The Creator Don't support type [Cast], /block/self_attention/core_attention/Cast_output_0 Don't support type [Select], /block/self_attention/core_attention/Where_output_0 Don't support type [While], /block/self_attention/core_attention/MatMul_1_output_0 Don't support type [While], /block/Add_output_0 2, 32 - 1 The Creator Don't support type [Cast], /block/post_attention_layernorm/Cast_output_0 2, 32 - 1 The Creator Don't support type [Cast], /block/post_attention_layernorm/Cast_1_output_0 2, 32 - 1 The Creator Don't support type [Cast], /final_layernorm/Cast_output_0 2, 32 - 1 The Creator Don't support type [Cast], /final_layernorm/Cast_1_output_0 Don't support type [While], /Gather_output_0 Don't support type [ArgMax], token_id