search

NVIDIA / TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
https://developer.nvidia.com/tensorrt
Apache License 2.0
10.89k stars 2.14k forks source link

node.output().size() == 1 && "TensorRT does not support the indices output in MaxPool!" failure of TensorRT 8.6 when running ONNX model on GPU RTX 3090 #3761

Open lhai37 opened 8 months ago

lhai37 commented 8 months ago

Description

I tried to run the attached ONNX model through trtexec, but it fails with the error: node.output().size() == 1 && "TensorRT does not support the indices output in MaxPool!". This is because MaxPool is being used in the model to produce 2 outputs: the values and indices. Upon checking the officially supported ONNX operators I see that MaxPool layer with indices output is not supported even in TensorRT 10 EA. Is there any plan to support this or workaround for this issue?

Environment

TensorRT Version: 8.6

NVIDIA GPU: RTX 3090

NVIDIA Driver Version: 545

CUDA Version: 11.6

CUDNN Version: 8.8

Operating System: Ubuntu 20.04 64-bit

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link: https://drive.google.com/file/d/1ZvhQiG9NT92CbVK7oAznBR_rAlF5AzzF/view?usp=sharing

Steps To Reproduce

Run trtexec --onnx=model.onnx

Commands or scripts: Run trtexec --onnx=model.onnx

Have you tried the latest release?: N/A

Can this model run on other frameworks? N/A

lix19937 commented 8 months ago

ref
https://docs.nvidia.com/deeplearning/tensorrt/operators/docs/Pooling.html https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Graph/Layers.html#ipoolinglayer

Computes a per-channel pooling using a sampling window on the input tensor into an output tensor. The supported sampling window shapes are 2-D or 3-D.
If you need 2 outputs of pooling layer, i think for now you’d need to use a plugin.

lhai37 commented 8 months ago

Is there any existing plugin for MaxPool with 2 outputs? I looked through the default plugins that are shipped with TensorRT and didn't see any there, but perhaps someone has already created this elsewhere.

lix19937 commented 8 months ago

You can write a custom plugin by yourself.

bs1119 commented 4 months ago

Is there any existing plugin for MaxPool with 2 outputs? I looked through the default plugins that are shipped with TensorRT and didn't see any there, but perhaps someone has already created this elsewhere.

Were you able to solve the issue? If so, can you please share the plugin? Thanks.

lix19937 commented 4 months ago

You can upload your torch network code fragment about maxpool.