phi2 使用 llm_demo 推理报错

quinlan-w commented 2 months ago

model path is ../../llm_export/phi2-int4/llm.mnn
### model name : Phi_2
The device support i8sdot:1, support fp16:1, support i8mm: 1
Can't open file:.tempcache
Load Cache file error.
load tokenizer
load tokenizer Done
### disk embedding is 1
load ../../llm_export/phi2-int4/llm.mnn ... Done!
main, 95, cost time: 12719.451172 ms
Prepare for resize opt Begin
Error: No encoding found for the sequence starting at position 2
Segmentation fault

附上模型转换的命令： python llm_export.py --path /data/zhiquan.wang/llm_models/phi-2/ --embed_bin --embed_bf16 --export_embed --export_token --export --type phi-2 --mnn_path phi2-int4 --onnx_path phi-2 /data/zhiquan.wang/code_llm/MNN/build/MNNConvert -f ONNX --modelFile phi-2/llm.onnx --MNNModel phi2-int4/llm.mnn --weightQuantBits=4 --saveExternalData

jxt1234 commented 2 months ago

tokenizer.txt 是否正确？

quinlan-w commented 2 months ago

tokenizer.txt 是否正确？

如何验证下是否正确？前几个是这样的

50295 50001
!
"
#
$
%

github-actions[bot] commented 1 week ago

Marking as stale. No activity in 60 days.

alibaba / MNN

phi2 使用 llm_demo 推理报错 #2881