search

NVIDIA-AI-IOT / torch2trt

An easy to use PyTorch to TensorRT converter
MIT License
4.6k stars 675 forks source link

Problem converting Keypoint R-CNN model #761

Open thebruce87m opened 2 years ago

thebruce87m commented 2 years ago

Hi,

I am trying to run the Keypoint R-CNN in NVIDIA Deepstream. I think to do that I need to convert pytorch model to a TRT model. The Pytorch model is here: https://pytorch.org/vision/stable/models/keypoint_rcnn.html

I run as following:

Clone the repo

git clone https://github.com/NVIDIA-AI-IOT/torch2trt.git

Run the docker

docker run \
--gpus all \
-it \
--rm \
--ipc=host \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
-v $(pwd):/code \
-w /code/ \
nvcr.io/nvidia/pytorch:22.06-py3

Build the plugin inside the docker

cd torch2trt/

python setup.py install

cmake -B build . && cmake --build build --target install && ldconfig

My Conversion Script:

/code/convert.py:

import torch
from torch2trt import torch2trt

import torchvision
from torchvision.models.detection.rpn import AnchorGenerator

def get_model(num_keypoints, weights_path=None):

    anchor_generator = AnchorGenerator(sizes=(32, 64, 128, 256, 512), aspect_ratios=(0.25, 0.5, 0.75, 1.0, 2.0, 3.0, 4.0))
    model = torchvision.models.detection.keypointrcnn_resnet50_fpn(pretrained=False,
                                                                   pretrained_backbone=True,
                                                                   num_keypoints=num_keypoints,
                                                                   num_classes = 2, # Background is the first class, object is the second class
                                                                   rpn_anchor_generator=anchor_generator)

    if weights_path:
        state_dict = torch.load(weights_path)
        model.load_state_dict(state_dict)        

    return model

model = get_model(num_keypoints = 2)

model = model.eval().cuda()

# create example data
x = torch.ones((1, 3, 224, 224)).cuda()

# convert to TensorRT feeding sample data as input
model_trt = torch2trt(model, [x])

Error

However, I get the following error:

python3 /code/convert.py 
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained_backbone' is deprecated since 0.13 and will be removed in 0.15, please use 'weights_backbone' instead.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights_backbone' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights_backbone=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights_backbone=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Warning: Encountered known unsupported method torch.Tensor.unbind
Warning: Encountered known unsupported method torch.Tensor.__iter__
Warning: Encountered known unsupported method torch.Tensor.unbind
Warning: Encountered known unsupported method torch.Tensor.__iter__
Warning: Encountered known unsupported method torch.Tensor.is_floating_point
Warning: Encountered known unsupported method torch.as_tensor
Warning: Encountered known unsupported method torch.as_tensor
Traceback (most recent call last):
  File "/code/convert.py", line 34, in <module>
    model_trt = torch2trt(model, [x])
  File "/opt/conda/lib/python3.8/site-packages/torch2trt-0.3.0-py3.8.egg/torch2trt/torch2trt.py", line 644, in torch2trt
    outputs = module(*inputs)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1148, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torchvision/models/detection/generalized_rcnn.py", line 83, in forward
    images, targets = self.transform(images, targets)
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1148, in _call_impl
    result = forward_call(*input, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/torchvision/models/detection/transform.py", line 129, in forward
    image = self.normalize(image)
  File "/opt/conda/lib/python3.8/site-packages/torchvision/models/detection/transform.py", line 157, in normalize
    return (image - mean[:, None, None]) / std[:, None, None]
  File "/opt/conda/lib/python3.8/site-packages/torch2trt-0.3.0-py3.8.egg/torch2trt/torch2trt.py", line 300, in wrapper
    converter["converter"](ctx)
  File "/opt/conda/lib/python3.8/site-packages/torch2trt-0.3.0-py3.8.egg/torch2trt/converters/getitem.py", line 30, in convert_tensor_getitem
    input_trt = input._trt
AttributeError: 'Tensor' object has no attribute '_trt'
root@f3da85188578:/code/torch2trt# 
root@f3da85188578:/code/torch2trt# 
root@f3da85188578:/code/torch2trt# 
root@f3da85188578:/code/torch2trt# python3 /code/convert.py 
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and will be removed in 0.15, please use 'weights' instead.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained_backbone' is deprecated since 0.13 and will be removed in 0.15, please use 'weights_backbone' instead.
  warnings.warn(
/opt/conda/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights_backbone' are deprecated since 0.13 and will be removed in 0.15. The current behavior is equivalent to passing `weights_backbone=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights_backbone=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
  warnings.warn(msg)
Warning: Encountered known unsupported method torch.Tensor.unbind
Warning: Encountered known unsupported method torch.Tensor.__iter__
Warning: Encountered known unsupported method torch.Tensor.unbind
Warning: Encountered known unsupported method torch.Tensor.__iter__
Warning: Encountered known unsupported method torch.Tensor.is_floating_point
Warning: Encountered known unsupported method torch.as_tensor
Warning: Encountered known unsupported method torch.as_tensor
Traceback (most recent call last):
  File "/code/convert.py", line 34, in <module>
    model_trt = torch2trt(model, [x])

What am I doing wrong?

thebruce87m commented 2 years ago

Looks like the same problem on #524