search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.68k stars 2.12k forks source link

trtexec --exportProfile profile files only have one layer of information, and the model structure cannot be restored. #3768

Closed 18liumin closed 5 months ago

18liumin commented 6 months ago

image

If there is torch.cat, there will be a problem that the model structure has only one layer.

zerollzeng commented 6 months ago

Because they are fused by TRT so you can only see a layer with some name like myelin_xxx?

18liumin commented 6 months ago

Because they are fused by TRT so you can only see a layer with some name like myelin_xxx?

engine.profile.json has {ForeignNode[encoder.fea_bn1.weight.../softmax/Softmax]}

zerollzeng commented 6 months ago

Yes this is the fused optimized graph.

ttyio commented 5 months ago

closing since no activity for more than 3 weeks per our policy, thanks all!