search

onnx / onnx-tensorrt

ONNX-TensorRT: TensorRT backend for ONNX
Apache License 2.0
2.91k stars 540 forks source link

Is there any plan to support SequenceConstruct? #863

Open DoGunKIM93 opened 2 years ago

DoGunKIM93 commented 2 years ago

Description

onnx-tensorrt/docs/operators.md still says don't support SequenceConstruct, SequenceAt, etc. So is any plan to support these Sequence ops?

Environment

TensorRT Version: 8.0.4 ONNX-TensorRT Version / Branch: main GPU Type: V100 Nvidia Driver Version: 470.103.01 CUDA Version: 11.6 CUDNN Version: 8.2.4 Operating System + Version: ubuntu 20.04 Python Version (if applicable): 3.8.12 TensorFlow + TF2ONNX Version (if applicable): PyTorch Version (if applicable): 1.11.0 Baremetal or Container (if container which image + tag): nvcr.io/nvidia/pytorch:22.04-py3

Relevant Files

Steps To Reproduce

kevinch-nv commented 2 years ago

There is no immediate plan to support sequence ops in TensorRT .

anhtu812 commented 10 months ago

I need one of three option to implement seq2seq decode model (use transformer):

  1. sequence ops
  2. IRecurrenceLayer support change size input when loop (time axis increase size when loop)
  3. Slice by iterator of loop (allocate max length and slice to use) @kevinch-nv Is there any plan to support seq2seq decode model?
borisfom commented 10 months ago

@kevinch-nv @rajeevsrao : Basically, most of the loops would require a sequence.

ryanli commented 4 months ago

If anyone gets to this issue while trying to convert an ONNX model to TensorRT: TorchDynamo as of torch 2.2.2 converts torch.cat to this op, and if you switch to TorchScript instead you won't have this issue for the conversion.