Closed thunder95 closed 1 year ago
FAILED TensorRT.trtexec [TensorRT v8601] [optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/model/layers.31/self_attn/Slice_6_output_0[Constant].../lm_head/MatMul]}.)
Tried to convert a llm to trt model, used pytorch 2.2.0 cpu, with the following parameters: org_dummy_input = make_train_dummy_input(128) output = torch.onnx.export(model, org_dummy_input, "baichuan2.onnx", verbose=False, do_constant_folding=False, opset_version=14, input_names=['input_ids', 'attention_mask'], output_names=['lm_head'], dynamic_axes={"input_ids": {0: "batch_size", 1: "sequence_length"}, "attention_mask": {0: "batch_size", 1: "sequence_length"}, "lm_head": {0: "batch_size"}})
This model is from https://github.com/baichuan-inc/Baichuan2. And this error is quite similar to this issue: https://github.com/NVIDIA/TensorRT/issues/2124. Please help me, thanks!
FAILED TensorRT.trtexec [TensorRT v8601] [optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[/model/layers.31/self_attn/Slice_6_output_0[Constant].../lm_head/MatMul]}.)
Tried to convert a llm to trt model, used pytorch 2.2.0 cpu, with the following parameters: org_dummy_input = make_train_dummy_input(128) output = torch.onnx.export(model, org_dummy_input, "baichuan2.onnx", verbose=False, do_constant_folding=False, opset_version=14, input_names=['input_ids', 'attention_mask'], output_names=['lm_head'],
dynamic_axes={"input_ids": {0: "batch_size", 1: "sequence_length"}, "attention_mask": {0: "batch_size", 1: "sequence_length"}, "lm_head": {0: "batch_size"}})
This model is from https://github.com/baichuan-inc/Baichuan2. And this error is quite similar to this issue: https://github.com/NVIDIA/TensorRT/issues/2124. Please help me, thanks!