search

wangzhaode / llm-export

llm-export can export llm model to onnx.
Apache License 2.0
190 stars 21 forks source link

修改代码以便可以从onnx转换为trt格式 #2

Open xiaobai52HZ opened 11 months ago

xiaobai52HZ commented 11 months ago

后续可以使用trtexec工具转换为trt格式 trtexec --onnx=./model.onnx --saveEngine=./trt/model.plan --optShapes=input_ids:1,attention_mask:1x1x1x1026,position_ids:1x1,past_key_values:32x2x1x32x1025x128 --minShapes=input_ids:1,attention_mask:1x1x1x1,position_ids:1x1,past_key_values:32x2x1x32x0x128 --maxShapes=input_ids:1024,attention_mask:1x1x1024x2049,position_ids:1x1024,past_key_values:32x2x1x32x1025x128 --device=1 --fp16

xiaobai52HZ commented 10 months ago

trtexec --onnx=./model.onnx --saveEngine=./trt/model.plan --optShapes=input_ids:1,attention_mask:1x1x1x1026,position_ids:1x1,past_key_values:32x2x1x32x1025x128 --minShapes=input_ids:1,attention_mask:1x1x1x1,position_ids:1x1,past_key_values:32x2x1x32x0x128 --maxShapes=input_ids:1024,attention_mask:1x1x1024x2049,position_ids:1x1024,past_key_values:32x2x1x32x1025x128 --device=1 --fp16

---原始邮件--- 发件人: @.> 发送时间: 2023年11月5日(周日) 下午2:04 收件人: @.>; 抄送: @.**@.>; 主题: Re: [wangzhaode/llm-export] 修改代码以便可以从onnx转换为trt格式 (PR #2)

how about elimate zero input (32x2x1x32x0x128)：https://github.com/torchpipe/LLM.TensorRT.Serve

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>