Closed WangHao311 closed 1 month ago
Killed means out of memory, maybe you can add --skip_slim
Thank you very much, “--skip_slim” is helpful. But by the way, how much memory is needed? My machine is 32GB.
you can do it offline
onnxslim ./qwen1.5-1.8b-onnx/llm.onnx ./qwen1.5-1.8b-onnx/slim.onnx
The offline onnxslim command is available, but how can I get the final file similar to this link? https://modelscope.cn/models/zhaode/Qwen1.5-1.8B-Chat-MNN/files
you can converte it offline too
mnnconvert -f ONNX --modelFile ./qwen1.5-1.8b-onnx/slim.onnx --MNNModel ./qwen1.5-1.8b-onnx/qwen-1.8b-int4.mnn --weightQuantBits 4 --weightQuantAsymmetric --saveExternalData
This's great, thank you!
代码: 2024-07-18 拉取的master分支最新代码;
主要环境: MNN==2.8.3 numpy==1.26.4 onnxruntime==1.15.1 torch==2.0.1 transformers==4.41.2 transformers_stream_generator==0.0.4 sentencepiece==0.1.99 onnxslim==0.1.32
转换命令:
python llm_export.py --path ./Qwen1.5-1.8B-Chat --export --export_embed --embed_bin --export_token --export_mnn --type Qwen1_5-1_8B-Chat --onnx_path ./qwen1.5-1.8b-onnx --mnn_path ./qwen1.5-1.8b-mnn
日志如下:
从日志看,在执行slim(onnx_model, output_model=onnx_model)的过程出错了。 @wangzhaode @inisis 请大佬帮忙看一下,我应该如何解决?