🐛 [Bug] DETR network conversion failed

vinhngx commented 2 years ago

Bug Description

DETR network obtained from:

model = torch.hub.load('facebookresearch/detr:main', 'detr_resnet50', pretrained=True)

Scripting doesn't work with PyTorch. So we're following the tracing path.

traced_model = torch.jit.trace(model, [torch.randn((128, 3, 224, 224)).to("cuda")], strict=False)

Conversion with/without fallback both failed:

import trtorch

# The compiled module will have precision as specified by "op_precision".
# Here, it will have FP16 precision.
trt_model = trtorch.compile(traced_model, {
    "inputs": [trtorch.Input((128, 3, 224, 224))],
    "enabled_precisions": {torch.half}, # Run with FP16
    "workspace_size": 1 << 20, 
    "torch_fallback":{"enabled": True}
})

Without fallback:

RuntimeError: [Error thrown at core/conversion/conversion.cpp:118] Expected converter to be true but got false
Unable to convert node: %44 : Tensor[] = aten::unbind(%samples, %36) # /opt/conda/lib/python3.8/site-packages/torch/_tensor.py:608:0 (conversion.AddLayer)
Schema: aten::unbind.int(Tensor(a) self, int dim=0) -> (Tensor[])
Converter for aten::unbind requested, but no such converter was found.
If you need a converter for this operator, you can try implementing one yourself
or request a converter: https://www.github.com/NVIDIA/TRTorch/issues

With fallback:

----> 5 trt_model = trtorch.compile(traced_model, {
      6     "inputs": [trtorch.Input((128, 3, 224, 224))],
      7     "enabled_precisions": {torch.half}, # Run with FP16

/opt/conda/lib/python3.8/site-packages/trtorch/_compiler.py in compile(module, compile_spec)
     79             "torch.jit.ScriptFunction currently is not directly supported, wrap the function in a module to compile")
     80 
---> 81     compiled_cpp_mod = trtorch._C.compile_graph(module._c, _parse_compile_spec(compile_spec))
     82     compiled_module = torch.jit._recursive.wrap_cpp_module(compiled_cpp_mod)
     83     return compiled_module

ValueError: vector::_M_range_insert

To Reproduce

Steps to reproduce the behavior:

Prototype here /mnt/nvdl/usr/vinhn/trtorch-perf-benchmar/DETR

Expected behavior

Environment

Build information about the TRTorch compiler can be found by turning on debug messages

TRTorch Version (e.g. 0.2.0):
PyTorch Version (e.g. 1.0):
CPU Architecture:
OS (e.g., Linux):
How you installed PyTorch (conda, pip, libtorch, source):
Build command you used (if compiling from source):
Are you using local sources or building from archives:
Python version:
CUDA version:
GPU models and configuration:
Any other relevant information:

Additional context

github-actions[bot] commented 2 years ago

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

github-actions[bot] commented 1 year ago

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

peri044 commented 1 year ago

1) We support aten::unbind operator now. So you could give it a try without fallback now. 2) If you still have some converter issues, can you try fallback with our latest 1.3 release ?

In my past (limited) experience with DETR, some model changes were required to ensure torchscript was in the right format that Torch-TRT could operate on. Not sure if they are still valid

Christina-Young-NVIDIA commented 1 year ago

This is a model feature request - whether or not we will support this model.

github-actions[bot] commented 1 year ago

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

pytorch / TensorRT