search

NVIDIA-AI-IOT / torch2trt

An easy to use PyTorch to TensorRT converter
MIT License
4.6k stars 676 forks source link

'Tensor' object has no attribute '_trt' #15

Open gavrin-s opened 5 years ago

gavrin-s commented 5 years ago

I get error AttributeError: 'Tensor' object has no attribute '_trt' when I meet MaxPool2d

/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/torch2trt.py in wrapper(*args, **kwargs) 95 96 #print('%s : %s' % (method.qualname, converter.name)) ---> 97 converter(ctx) 98 99 # convert to None so conversion will fail for unsupported layers

/usr/local/lib/python3.6/dist-packages/torch2trt-0.0.0-py3.6.egg/torch2trt/converters/MaxPool2d.py in convert_MaxPool2d(ctx) 21 22 layer = ctx.network.add_pooling( ---> 23 input=input._trt, type=trt.PoolingType.MAX, window_size=kernel_size) 24 layer.stride = stride 25 layer.padding = padding

AttributeError: 'Tensor' object has no attribute '_trt'

mtheise8 commented 5 years ago

I'm also getting this error when I meet AvgPool2d trying to convert the ESPNetV2 model here: https://github.com/sacmehta/EdgeNets

jaybdub commented 5 years ago

Hi All,

This is most likely because there Is a layer converter that has not been implemented. It is likely the layer before the one that threw the error (because the _trt was not set by the previous converter).

We’re focused on supporting the models in the README, but the coverage may increase over time.

@gavrin-s are you able to share the model you’re attempting to convert?

FYI, You can see the list of registered converters by typing

‘’’python import torch2trt

print(torch2trt.CONVERTERS) ‘’’

It may be possible to add support for the unsupported layers by using the ‘’@tensorrt_converter’’ method described in the README.md.

Please let me know if this helps or you have any other questions.

Best, John

vfdev-5 commented 5 years ago

HI @jaybdub ,

I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.

Click to see the log

``` --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) in ----> 1 model_trt = torch2trt.torch2trt(model_w, [data], fp16_mode=True) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/torch2trt.py in torch2trt(module, inputs, input_names, output_names, log_level, max_batch_size, fp16_mode, max_workspace_size) 250 ctx.add_inputs(inputs, input_names) 251 --> 252 outputs = module(*inputs) 253 254 if not isinstance(outputs, tuple): ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 491 result = self._slow_forward(*input, **kwargs) 492 else: --> 493 result = self.forward(*input, **kwargs) 494 for hook in self._forward_hooks.values(): 495 hook_result = hook(self, input, result) in forward(self, x) 4 self.model = model 5 def forward(self, x): ----> 6 return model(x)['out'] ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 491 result = self._slow_forward(*input, **kwargs) 492 else: --> 493 result = self.forward(*input, **kwargs) 494 for hook in self._forward_hooks.values(): 495 hook_result = hook(self, input, result) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torchvision/models/segmentation/_utils.py in forward(self, x) 20 result = OrderedDict() 21 x = features["out"] ---> 22 x = self.classifier(x) 23 x = F.interpolate(x, size=input_shape, mode='bilinear', align_corners=False) 24 result["out"] = x ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 491 result = self._slow_forward(*input, **kwargs) 492 else: --> 493 result = self.forward(*input, **kwargs) 494 for hook in self._forward_hooks.values(): 495 hook_result = hook(self, input, result) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/container.py in forward(self, input) 90 def forward(self, input): 91 for module in self._modules.values(): ---> 92 input = module(input) 93 return input 94 ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs) 491 result = self._slow_forward(*input, **kwargs) 492 else: --> 493 result = self.forward(*input, **kwargs) 494 for hook in self._forward_hooks.values(): 495 hook_result = hook(self, input, result) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torchvision/models/segmentation/deeplabv3.py in forward(self, x) 90 for conv in self.convs: 91 res.append(conv(x)) ---> 92 res = torch.cat(res, dim=1) 93 return self.project(res) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/torch2trt.py in wrapper(*args, **kwargs) 95 96 #print('%s : %s' % (method.__qualname__, converter.__name__)) ---> 97 converter(ctx) 98 99 # convert to None so conversion will fail for unsupported layers ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/converters/cat.py in convert_cat(ctx) 12 13 output = ctx.method_return ---> 14 trt_inputs = [i._trt for i in inputs] 15 16 layer = ctx.network.add_concatenation(inputs=trt_inputs) ~/miniconda3/envs/py35/lib/python3.5/site-packages/torch2trt-0.0.0-py3.5.egg/torch2trt/converters/cat.py in (.0) 12 13 output = ctx.method_return ---> 14 trt_inputs = [i._trt for i in inputs] 15 16 layer = ctx.network.add_concatenation(inputs=trt_inputs) AttributeError: 'Tensor' object has no attribute '_trt' ```
Any ideas why it does not work ? Packages Versions
``` # packages in environment at /root/miniconda3/envs/py35: # # Name Version Build Channel _libgcc_mutex 0.1 main attrs 19.1.0 pypi_0 pypi backcall 0.1.0 pypi_0 pypi blas 1.0 mkl bleach 3.1.0 pypi_0 pypi ca-certificates 2019.5.15 0 certifi 2018.8.24 py35_1 cffi 1.11.5 py35he75722e_1 cudatoolkit 10.0.130 0 decorator 4.4.0 pypi_0 pypi defusedxml 0.6.0 pypi_0 pypi entrypoints 0.3 pypi_0 pypi freetype 2.9.1 h8a8886c_1 intel-openmp 2019.4 243 ipykernel 5.1.1 pypi_0 pypi ipython 7.6.1 pypi_0 pypi ipython-genutils 0.2.0 pypi_0 pypi ipywidgets 7.5.0 pypi_0 pypi jedi 0.14.1 pypi_0 pypi jinja2 2.10.1 pypi_0 pypi jpeg 9b h024ee3a_2 jsonschema 3.0.1 pypi_0 pypi jupyter 1.0.0 pypi_0 pypi jupyter-client 5.3.1 pypi_0 pypi jupyter-console 6.0.0 pypi_0 pypi jupyter-core 4.5.0 pypi_0 pypi libedit 3.1.20181209 hc058e9b_0 libffi 3.2.1 hd88cf55_4 libgcc-ng 9.1.0 hdf63c60_0 libgfortran-ng 7.3.0 hdf63c60_0 libpng 1.6.37 hbc83047_0 libstdcxx-ng 9.1.0 hdf63c60_0 libtiff 4.0.10 h2733197_2 markupsafe 1.1.1 pypi_0 pypi mistune 0.8.4 pypi_0 pypi mkl 2019.4 243 nbconvert 5.5.0 pypi_0 pypi nbformat 4.4.0 pypi_0 pypi ncurses 6.1 he6710b0_1 ninja 1.8.2 py35h6bb024c_1 notebook 6.0.0 pypi_0 pypi numpy 1.14.2 py35hdbf6ddf_0 olefile 0.46 py35_0 openssl 1.0.2s h7b6447c_0 pandocfilters 1.4.2 pypi_0 pypi parso 0.5.1 pypi_0 pypi pexpect 4.7.0 pypi_0 pypi pickleshare 0.7.5 pypi_0 pypi pillow 5.2.0 py35heded4f4_0 pip 10.0.1 py35_0 prometheus-client 0.7.1 pypi_0 pypi prompt-toolkit 2.0.9 pypi_0 pypi ptyprocess 0.6.0 pypi_0 pypi pycparser 2.19 py35_0 pygments 2.4.2 pypi_0 pypi pyrsistent 0.15.3 pypi_0 pypi python 3.5.6 hc3d631a_0 python-dateutil 2.8.0 pypi_0 pypi pytorch 1.1.0 py3.5_cuda10.0.130_cudnn7.5.1_0 pytorch pyzmq 18.0.2 pypi_0 pypi qtconsole 4.5.2 pypi_0 pypi readline 7.0 h7b6447c_5 send2trash 1.5.0 pypi_0 pypi setuptools 40.2.0 py35_0 six 1.11.0 py35_1 sqlite 3.29.0 h7b6447c_0 terminado 0.8.2 pypi_0 pypi testpath 0.4.2 pypi_0 pypi tk 8.6.8 hbc83047_0 torch2trt 0.0.0 pypi_0 pypi torchvision 0.3.0 py35_cu10.0.130_1 pytorch tornado 6.0.3 pypi_0 pypi traitlets 4.3.2 pypi_0 pypi wcwidth 0.1.7 pypi_0 pypi webencodings 0.5.1 pypi_0 pypi wheel 0.31.1 py35_0 widgetsnbextension 3.5.0 pypi_0 pypi xz 5.2.4 h14c3975_4 zlib 1.2.11 h7b6447c_3 zstd 1.3.7 h0b5b093_0 ```
lawrencekiba commented 5 years ago

HI @jaybdub ,

I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic.

Click to see the log

Any ideas why it does not work ?

Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

lawrencekiba commented 5 years ago

HI @jaybdub , I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic. Click to see the log Any ideas why it does not work ? Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

Tried running commands in https://github.com/NVIDIA-AI-IOT/torch2trt#convert and encountered similar errors too.

kawa23 commented 5 years ago

Hi All,

This is most likely because there Is a layer converter that has not been implemented. It is likely the layer before the one that threw the error (because the _trt was not set by the previous converter).

We’re focused on supporting the models in the README, but the coverage may increase over time.

@gavrin-s are you able to share the model you’re attempting to convert?

FYI, You can see the list of registered converters by typing

‘’’python import torch2trt

print(torch2trt.CONVERTERS) ‘’’

It may be possible to add support for the unsupported layers by using the ‘’@tensorrt_converter’’ method described in the README.md.

Please let me know if this helps or you have any other questions.

Best, John

Hi, @jaybdub
I can't find the Upsample and LeakyReLU at torch2trt.CONVERTERS, is it still not supported yet?

As you said, it can be solved by using the @tensorrt_converter, but I don't know how to start, could you show a demo, thanks. BTW, I installed torch2trt without plugins.

jaybdub commented 5 years ago

Hi kawa23,

Thanks for reaching out.

We have implemented the interpolate plugin, which can accomplish Upsampling. You will need to install torch2trt with plugins as described in the README.

For LeakyReLU, the following converter should work

import numpy as np
import tensorrt
import torch2trt

logger = trt.Logger(trt.Logger.INFO)
trt.init_libnvinfer_plugins(logger, '')

@torch2trt.tensorrt_converter('torch.nn.functional.leaky_relu')
def convert_leaky_relu(ctx):
    input = ctx.method_args[0]
    output = ctx.method_return

    if len(ctx.method_args) > 1:
        negative_slope = ctx.method_args[1]
    elif 'negative_slope' in ctx.method_kwargs:
        negative_slope = ctx.method_kwargs['negative_slope']

    registry = trt.get_plugin_registry()

    creator = [c for c in registry.plugin_creator_list if c.name == 'LReLU_TRT'][0]
    lrelu_slope_field = trt.PluginField("neg_slope", np.array([negative_slope], dtype=np.float32), trt.PluginFieldType.FLOAT32)
    field_collection = trt.PluginFieldCollection([lrelu_slope_field])
    plugin = creator.create_plugin(name='LReLU_TRT', field_collection=field_collection)

    layer = ctx.network.add_plugin_v2(inputs=[input._trt], plugin=plugin)

    output._trt = layer.get_output(0)

After executing the above code, you should be able to convert the model as described in the README.

Please let me know if this works for you or if you run into any further issues. I'd be happy to take a look at your model if you continue to run into problems.

Best, John

cloudhan commented 5 years ago

OK, after reading the code, I finally figure out what happens under the hood. There is a ConversionContext, when enter it, some pytorch's methods are replaced with wrapped methods, the wrapped methods do:

  1. run the original method (to driven the conversion process)
  2. do the conversion

problem happens when you encounter some unhooked method, they will silently fail the latter conversion, some part is unconnected. Main issue is not the failure in @gavrin-s 's case, but is, you don't know what missing hook trigger the failure, only looking into the original pytorch code will give you the insight...

jaybdub commented 5 years ago

@cloudhan Your understanding is correct. Admittedly this makes it a bit harder to debug a missing converter, and currently in some cases reading the code may be necessary.

One tip that may help, is to check the grad_fn of the tensor which is missing the _trt attribute. This is set for any non-leaf tensor requiring gradient. I believe you can check this by

  1. Attempt conversion (should throw error)

    model_trt = torch2trt(model, [data])

  2. Launch debugger post-mortem

    import pdb
pdb.pm()

  3. Print grad_fn of tensor without _trt attribute.

    p input.grad_fn

  4. Manually search for torch method which corresponds to printed backward method

Perhaps we could log this automatically upon missing _trt error, to provide an extra hint when conversion fails.

Please let me know if this helps or you have any other questions or feedback.

Best, John

wangguojun2018 commented 5 years ago

HI @jaybdub , I'm trying to reproduce image segmentation example of this tutorial (Deeplabv3 model) and I have the following issue as in this topic. Click to see the log Any ideas why it does not work ? Packages Versions

I have a similar error when working to replicate the image classification example

Traceback (most recent call last): File "conversionog.py", line 14, in <module> model_trt = torch2trt(model, [data]) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 252, in torch2trt outputs = module(*inputs) File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "/usr/local/lib/python3.5/dist-packages/torchvision-0.3.0a0+6a834e9-py3.5-linux-x86_64.egg/torchvision/models/resnet.py", line 208, in forward File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 539, in __call__ result = self.forward(*input, **kwargs) File "Desktop/torch2trt/torch2trt/torch2trt.py", line 97, in wrapper converter(ctx) File "Desktop/torch2trt/torch2trt/converters/Linear.py", line 12, in convert_Linear layer = ctx.network.add_shuffle(input._trt) AttributeError: 'Tensor' object has no attribute '_trt'

I have the same problem, I open a issue in 124

kausainshah commented 4 years ago

AttributeError: 'Tensor' object has no attribute '_trt' Even using the Convert script provided in README.md for alexnet

azuryl commented 2 years ago

@jaybdub

does torch2trt support nn.Parameter

when I covert efficientnet in https://github.com/shariqfarooq123/AdaBins/blob/2fb686a66a304f0a719bc53d77412460af97fd61/models/layers.py#L19 positional_encodings1 = self.positional_encodings[:embeddings1_shape, :]#.T.unsqueeze(0) File "/home/delight-gpu/project/torch2trt/torch2trt/torch2trt.py", line 300, in wrapper converter"converter" File "/home/delight-gpu/project/torch2trt/torch2trt/converters/getitem.py", line 30, in convert_tensor_getitem input_trt = input._trt AttributeError: 'Parameter' object has no attribute '_trt'

the self.positional_encodings = nn.Parameter(torch.rand(500, embedding_dim), requires_grad=True)