search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.13k stars 2.07k forks source link

A Simple Engine Build Fail on TensorRT 8.6 #3930

Open Aktcob opened 3 weeks ago

Aktcob commented 3 weeks ago

Hi, I build a simple onnx model on Tensorrt8.6. And I get an error: mhafusion.cpp:344: DCHECK(fc1 && fc2 && softmax) failed. Could not find any implementation for node {ForeignNode[onnx::MatMul_46 + (Unnamed Layer 1) [Shuffle].../_reduce_mlp/_mlp/_mlp.3/Add]}. ERROR: 10: [optimizer.cpp::computeCosts::3869] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[onnx::MatMul_46 + (Unnamed Layer 1) [Shuffle].../_reduce_mlp/_mlp/_mlp.3/Add]}.)

debug.zip

geraldstanje commented 3 weeks ago

what datatype is matmul_46?

lix19937 commented 2 weeks ago

@Aktcob Try to use trt10.01.

EmilioZhao commented 1 week ago

What's the root cause of this issue? We've met the same issue and cannot upgrade TensorRT from 8.6 to 10.01 in production environment. Are there any other solutions?

lix19937 commented 1 week ago

Node {ForeignNode[onnx::MatMul_46 + (Unnamed Layer* 1) [Shuffle].../_reduce_mlp/_mlp/_mlp.3/Add]} include many nodes that handle by trt internal Myelin-compiler, can you upload the full build log of trtexec with --verbose. @Aktcob