Closed xhzheng1895 closed 1 month ago
The device support i8sdot:0, support fp16:0, support i8mm: 0 Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [01:16<00:00, 10.95s/it] # senetencepiece tokenier ============= Diagnostic Run torch.onnx.export version 2.0.1+cu117 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ======================== +--------------------+-----------------------------------------+------------------------------+ | Model Name | embedding.onnx | Op Set: 15 | +--------------------+-----------------------------------------+------------------------------+ | Model Info | Original Model | Slimmed Model | +--------------------+-----------------------------------------+------------------------------+ | IN: input_ids | int64: ('length',) | int64: ('length',) | | OUT: inputs_embeds | float32: ('Reshapeinputs_embeds_dim_0', | float32: ('length', 1, 4096) | | | 'Reshapeinputs_embeds_dim_1', | | | | 'Reshapeinputs_embeds_dim_2') | | +--------------------+-----------------------------------------+------------------------------+ | Constant | 1 | 0 | | Gather | 1 | 1 | | Reshape | 1 | 1 | +--------------------+-----------------------------------------+------------------------------+ | Model Size | 1016.00 MB | 1016.00 MB | +--------------------+-----------------------------------------+------------------------------+ | Elapsed Time | 18.78 s | +--------------------+-----------------------------------------+------------------------------+ Don't has bizCode, use MNNTest for default Start to Convert Other Model Format To MNN Model..., target version: 2.8 [14:38:05] :46: ONNX Model ir version: 10 [14:38:05] :47: ONNX Model opset version: 15 Start to Optimize the MNN Net... inputTensors : [ input_ids, ] outputTensors: [ inputs_embeds, ] Converted Success! ============= Diagnostic Run torch.onnx.export version 2.0.1+cu117 ============= verbose: False, log level: Level.ERROR ======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ======================== +-------------------+---------------------+---------------------+ | Model Name | lm.onnx | Op Set: 15 | +-------------------+---------------------+---------------------+ | Model Info | Original Model | Slimmed Model | +-------------------+---------------------+---------------------+ | IN: hidden_states | float32: (1, 4096) | float32: (1, 4096) | | OUT: logits | float32: (1, 65024) | float32: (1, 65024) | +-------------------+---------------------+---------------------+ | MatMul | 1 | 1 | +-------------------+---------------------+---------------------+ | Model Size | 1016.00 MB | 1016.00 MB | +-------------------+---------------------+---------------------+ | Elapsed Time | 16.32 s | +-------------------+---------------------+---------------------+ Don't has bizCode, use MNNTest for default Start to Convert Other Model Format To MNN Model..., target version: 2.8 [14:39:07] :46: ONNX Model ir version: 10 [14:39:07] :47: ONNX Model opset version: 15 Killed
cmd line: python3 llm_export.py --type chatglm2-6b \ --path ../chatglm2-6b \ --export_split --export_token --export_mnn \ --onnx_path ./chatglm2-6b-int4-onnx \ --mnn_path ./chatglm2-6b-int4-mnn python version:3.9.19 transformers version:4.36.2(使用其他版本遇到别的问题) 其他包都follow requirement.txt
cmd line: python3 llm_export.py --type chatglm2-6b \ --path ../chatglm2-6b \ --export_split --export_token --export_mnn \ --onnx_path ./chatglm2-6b-int4-onnx \ --mnn_path ./chatglm2-6b-int4-mnn python version:3.9.19 transformers version:4.36.2(使用其他版本遇到别的问题) 其他包都follow requirement.txt