Error in initliaisation

[melezz]

hello!

When executing the following code I get an initialisation error

modelz = torchvision.models.alexnet(pretrained=True).eval().cuda() x = torch.ones((1,3,224,224)).cuda() model_trt = torch2trt(modelz, [x])

traceback:

Traceback (most recent call last): File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/main.py", line 57, in machine = stateMachine(new_video_FPS, weights) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 584, in init self.detector = Detector(weights,True) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 460, in init model_trt = torch2trt(modelz, [x]) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 244, in torch2trt builder.create_network() as network, ConversionContext(network) as ctx: File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 138, in init for method, converter in converters.items() File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 116, in init self.method_impl = copy(eval(method)) File "/usr/lib/python2.7/copy.py", line 96, in copy return _reconstruct(x, rv, 0) File "/usr/lib/python2.7/copy.py", line 329, in _reconstruct y = callable(args) File "/usr/lib/python2.7/copy_reg.py", line 93, in newobj return cls.new(cls, args) TypeError: instancemethod expected at least 2 arguments, got 0

Any ideas? thanks!

cheers

[jaybdub]

Hi melezz,

Thanks for reaching out! Could you clarify the following

1. Which version of Python are you using?
2. Which version of PyTorch are you using?

Best, John

[melezz]

Hello John,

1. Pytorch 1.0.0
2. Python 2.7

Both installed on Tegra on a Nvidia Xavier

cheers

[jaybdub]

Okay thanks. I think this is related to issue https://github.com/NVIDIA-AI-IOT/torch2trt/issues/4 for Python 2.7.

Could you try removing the copy() call on this line

https://github.com/NVIDIA-AI-IOT/torch2trt/blob/b7e87ca3f1701ee37c2c64bbce3cb10e2d71dccf/torch2trt/torch2trt.py#L116

And see if that fixes?

Please let me know if it does or doesn't or if you have any questions.

Best, John

[melezz]

Hi John, thanks for the answer. It did work but it looks like the converter still has a problem...I checked the code and it is exactly as it is in the readme example:

Traceback (most recent call last): File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/main.py", line 57, in machine = stateMachine(new_video_FPS, weights) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 584, in init self.detector = Detector(weights,True) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 460, in init model_trt = torch2trt(modelz, [x]) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 244, in torch2trt builder.create_network() as network, ConversionContext(network) as ctx: File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 143, in enter hook.enter() File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 123, in enter self._set_method(attach_converter(self.ctx, self.method_impl, self.converter)) AttributeError: 'ConversionHook' object has no attribute 'method_impl'

[jaybdub]

Hi melezz,

Just to clarify, did you change the line from

self.method_impl = copy(eval(method)) 

to

self.method_impl = eval(method)

Or remove the line entirely? My guess (since method_impl is missing) is that perhaps the entire line was removed.

Please let me know if this works for you and I will work on evaluating / patching.

Best, John

[melezz]

Hello John,

Apologies, I misunderstood it. Now after the change I still get an error, maybe due to tensorrt version?

My installed versrion: tensorrt (5.0.6.3)

New error: Traceback (most recent call last): File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/main.py", line 57, in machine = stateMachine(new_video_FPS, weights) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 584, in init self.detector = Detector(weights,True) File "/home/nvidia/Workspace/SCENARIO_3/tfl-car/aux_func.py", line 460, in init model_trt = torch2trt(modelz, [x]) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 267, in torch2trt return TRTModule(engine, ctx.input_names, ctx.output_names, final_shapes) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 182, in init self._register_state_dict_hook(TRTModule._on_state_dict) File "/home/nvidia/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 518, in getattr type(self).name, name)) AttributeError: 'TRTModule' object has no attribute '_register_state_dict_hook'

Then, I run test.py under torch2trt folder and I get this: Traceback (most recent call last): File "test.py", line 100, in max_error, fps, fps_trt, ms, ms_trt = run(test) File "test.py", line 25, in run module_trt = torch2trt(module, inputs, **self.torch2trt_kwargs) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 257, in torch2trt ctx.mark_outputs(outputs, output_names) File "/usr/local/lib/python2.7/dist-packages/torch2trt-0.0.0-py2.7.egg/torch2trt/torch2trt.py", line 172, in mark_outputs trt_tensor = torch_output._trt AttributeError: 'Tensor' object has no attribute '_trt'

Thanks!

cheers

[jaybdub]

Hi Melezz,

The first error AttributeError: 'TRTModule' object has no attribute '_register_state_dict_hook' is likely indicative of a PyTorch version mismatch. I have not tested this repository against PyTorch1.0, but only 1.1. Perhaps this feature was introduced then.

Are you able to test against PyTorch 1.1?

Best, John

[melezz]

Hi John, thanks a lot, you were right. I still get the other error but as I read in other thread it is due to the lack of support to specific models (mine is a unet implementation). Please let me know if you come up with a method to include new models functionality. Thanks!

[jaybdub]

Hi melezz,

Could you share the UNet implementation that you're interested in using?

Some models can be very difficult to convert (custom ops, unsupported layers, etc.). But it's possible there may be just one or two missing converters, which would be relatively easy to address.

Best, John

[jaybdub]

FYI, not sure if this is the model you're using, but i've added layers to support https://github.com/milesial/Pytorch-UNet

Only sigmoid and pad converters were missing.

You can convert the model by

model = UNet(n_channels=3, n_classes=1).cuda().half().eval()
data = torch.zeros((1, 3, WIDTH, HEIGHT)).cuda().half()

model_trt = torch2trt(model, [data], fp16_mode=True)

Speedup on Xavier with my test was about 3X vs plain PyTorch! Let me know if this works for you or you run into any issues.

Best, John

[melezz]

Hello John,

I can't share my model, but I can share the layers we are using:

• Already implemented: Conv2d,ReLU,BatchNorm2d, MxPool2d
• Implemented in your last commit: ZeroPad2d (I guess), Sigmoid
• Not implmented: Upsample

I think that Upsample is the most problematic, I tried to do the converter myself but couldn't find an appropiate layer in tensorrt (deconv would be the closest?)

I would be more than happy to use the plugin if the converter is ready!

thanks John, cheers