Open Justin020718 opened 3 months ago
Here is my changes in "naive_converters.py":
Despite unpleasant, I have to change like that otherwise I'll encounter a "ValueError: len() should return >= 0" issue.
The module works well in pytorch.
I've generated trt engine successfully through ONNX.
@Justin020718 ,
Thanks for reaching out. Apologies for the delay.
Glad to hear ONNX worked for you. FYI, not sure if this is what you did, but you can run torch2trt(..., use_onnx=True)
for convenience.
In case the issue re-appears / I need to reproduce, is this a publicly available model?
Best, John
@jaybdub Thank you for your reply. https://github.com/microsoft/DCVC/pull/18 I will try it later
@jaybdub Forgot to say, to reproduce my issue, you might need to add some converters in "native_converters.py"
@tensorrt_converter('torch.nn.functional.pixel_shuffle')
def convert_PixelShuffle(ctx):
input = ctx.method_args[0]
scale_factor = ctx.method_args[1]
input_trt = add_missing_trt_tensors(ctx.network, [input])[0]
output = ctx.method_return
batch_size, in_channels, in_height, in_width = input.shape
assert scale_factor >= 1
out_channels = in_channels // (scale_factor * scale_factor)
out_height = in_height * scale_factor
out_width = in_width * scale_factor
layer_1 = ctx.network.add_shuffle(input_trt)
layer_1.reshape_dims = (out_channels, scale_factor, scale_factor, in_height, in_width)
layer_2 = ctx.network.add_shuffle(layer_1.get_output(0))
layer_2.first_transpose = (0, 3, 1, 4, 2)
layer_2.reshape_dims = (batch_size, out_channels, out_height, out_width)
output._trt = layer_2.get_output(0)
def _set_layer_precision(ctx, layer):
# Supported TRT precisions as given by torch2trt_kwargs.
INT8_MODE = "int8_mode"
FP16_MODE = "fp16_mode"
# Check that args exist as expected in torch2trt_kwargs.
trt_kwargs = ctx.torch2trt_kwargs
assert INT8_MODE in trt_kwargs
assert FP16_MODE in trt_kwargs
is_int8 = trt_kwargs.get(INT8_MODE, False)
is_fp16 = trt_kwargs.get(FP16_MODE, False)
if is_int8:
layer.precision = trt.int8
layer.set_output_type(0, trt.int8)
elif is_fp16:
layer.precision = trt.float16
layer.set_output_type(0, trt.float16)
@tensorrt_converter('torch.zeros')
def convert_zeros(ctx):
tensor = ctx.method_return
# Implementation copied from add_trt_constant.
shape = tuple(tensor.shape[1:])
array = tensor[0].detach().cpu().numpy()
layer = ctx.network.add_constant(shape, array)
_set_layer_precision(ctx, layer)
tensor._trt = layer.get_output(0)
before operator converting, some input tensors had attributes called '_trt'
To deal with it, I've deleted these incorrect '_trt' attributes manually in related converter function in “naive_converters.py”.
However, I've encountered sone errors when doing things as follows, and it says:
Of course, the TRT module was not successfully generated, and when I try to save its state dict, it says:
Finally, here's my environment: pytorch 2.3.0+cu121 torch2trt 0.5.0 tensorRT 8.6.1.6 (Also tried tensorRT 10.0.1.6, after renaming some covnerting method, I have the same issue)
Since it's my first issue on Github, I really appreciate it if someone could help, or offer some clue to solve it, please!!!!!!!!!!!