how can i use group convolution ?

[root12321]

my kernel size is [256,1,1,4,4],and my input is [1,256,24,24],and i want to use group convolution ,and the groups is 256, i want to the output is [1,256,21,21],how can i do it ? any suggestion? i am using python tensorrt API @rmccorm4

[rmccorm4]

Hi @root12321,

I'm not quite sure how to do this off the top of my head - Try playing around with the num_groups variable in your convolution layer: https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/infer/Graph/Layers.html#tensorrt.IConvolutionLayer

[root12321]

this is my code

##cls_search_relu.get_output(0)=[1,256,24,24],cls_kernel_reshape.get_output(0)=[256,1,1,4,4]
cls_F_2d = network.add_convolution(input=cls_search_relu.get_output(0),num_output_maps=1,kernel_shape=(4,4), kernel=cls_kernel_reshape.get_output(0),bias=trt.Weights())
cls_F_2d.num_groups=256
cls_F_2d.stride = (1, 1)

but there is a error

Traceback (most recent call last):
  File "alxenet-onnx.py", line 464, in <module>
    build_engine(model.state_dict())
  File "alxenet-onnx.py", line 448, in build_engine
    populate_network(network, weights)
  File "alxenet-onnx.py", line 350, in populate_network
    cls_F_2d = network.add_convolution(input=cls_search_relu.get_output(0), num_output_maps=1,kernel_shape=(4,4), kernel=cls_kernel_reshape.get_output(0),bias=trt.Weights())
TypeError: add_convolution(): incompatible function arguments. The following argument types are supported:
    1. (self: tensorrt.tensorrt.INetworkDefinition, input: tensorrt.tensorrt.ITensor, num_output_maps: int, kernel_shape: tensorrt.tensorrt.DimsHW, kernel: tensorrt.tensorrt.Weights, bias: tensorrt.tensorrt.Weights) -> tensorrt.tensorrt.IConvolutionLayer

Invoked with: <tensorrt.tensorrt.INetworkDefinition object at 0x7f8981714030>; kwargs: num_output_maps=1, kernel=<tensorrt.tensorrt.ITensor object at 0x7f89817169d0>, kernel_shape=(4, 4), input=<tensorrt.tensorrt.ITensor object at 0x7f898170b110>, bias=<tensorrt.tensorrt.Weights object at 0x7f8981716a08>

@rmccorm4

[rmccorm4]

It looks like the expected input types are:

TypeError: add_convolution(): incompatible function arguments. The following argument types are supported:

(input: tensorrt.tensorrt.ITensor, 
num_output_maps: int, 
kernel_shape: tensorrt.tensorrt.DimsHW, 
kernel: tensorrt.tensorrt.Weights, 
bias: tensorrt.tensorrt.Weights)

But you provided:

input=<tensorrt.tensorrt.ITensor object at 0x7f898170b110>, 
num_output_maps=1, 
kernel_shape=(4, 4), 
kernel=<tensorrt.tensorrt.ITensor object at 0x7f89817169d0>, 
bias=<tensorrt.tensorrt.Weights object at 0x7f8981716a08>

If I were to guess I'd say this is because the kernel parameter expects tensorrt.tensorrt.Weights but you provided tensorrt.ITensor.

That being said, I'm not quite sure how to dynamically get the weights object from the layer - though I don't think that actually makes sense. I don't think you'd want to set the weights of your conv layer to your previous layer, they'd already be pre-defined, either hard-coded or taken from your trained model.

If you've already created / copied the weights for that layer somewhere in your code, you should be able to use that.

[root12321]

when i use the code to convert the type :

cls_F_2d = network.add_convolution(input=cls_search_relu.get_output(0), num_output_maps=1,kernel_shape=(4,4), kernel=trt.Weights(cls_kernel_reshape.get_output(0)),bias=trt.Weights())
cls_F_2d.num_groups=256
cls_F_2d.stride = (1, 1)

it has e error

Traceback (most recent call last):
  File "alxenet-onnx.py", line 464, in <module>
    build_engine(model.state_dict())
  File "alxenet-onnx.py", line 448, in build_engine
    populate_network(network, weights)
  File "alxenet-onnx.py", line 350, in populate_network
    cls_F_2d = network.add_convolution(input=cls_search_relu.get_output(0), num_output_maps=1,kernel_shape=(4,4), kernel=trt.Weights(cls_kernel_reshape.get_output(0)),bias=trt.Weights())
TypeError: __init__(): incompatible constructor arguments. The following argument types are supported:
    1. tensorrt.tensorrt.Weights(type: tensorrt.tensorrt.DataType = DataType.FLOAT)
    2. tensorrt.tensorrt.Weights(a: array)

Invoked with: <tensorrt.tensorrt.ITensor object at 0x7f439605e9d0>

how can i convert tensorrt.ITensor to tensorrt.tensorrt.Weights @rmccorm4

[rmccorm4]

Hi @root12321,

Since TensorRT is a platform for inference, not training, you need to plug-in your trained weights from somewhere else, not fetch them from your previous TensorRT layer.

If you've trained this model in another framework, you can export the weights from there as a numpy array, or just hard-code your own numpy array, and then pass in those weights to the kernel argument.

# Either taken from trained model in another framework like PyTorch/TF, or hard-coded
my_conv_weights = np.array(...)

cls_F_2d = network.add_convolution(
    input=cls_search_relu.get_output(0), 
    num_output_maps=1,
    kernel_shape=(4,4), 
    kernel=my_conv_weights,   # numpy_array can be used directly as tensorrt.Weights
    bias=trt.Weights()
)

[root12321]

but in my pytorch code,this convolution kernel is not trained weights ,it is calculated as a weights , that is say it is dynamic ,this is the code in pytorch :

    batch = int(kernel.size(0))
    channel = int(kernel.size(1))
    x = x.view(1, batch*channel, int(x.size(2)), int(x.size(3)))
    ##this kernel is the weights for convolution 
    kernel = kernel.view(batch*channel, 1, int(kernel.size(2)), int(kernel.size(3)))
    out = F.conv2d(x, kernel, groups=batch*channel)

so how can i load a dynamic weights in the tensorrt convolution kernel? @rmccorm4

[rmccorm4]

Sorry @root12321, I'm not sure how to help with that. Maybe someone else can chime in on this.

You'll probably have an easier time with this by exporting your PyTorch model to ONNX and using the ONNX parser to convert to TensorRT.

[rmccorm4]

Closing since no response.

[GabrieldeBlois]

Hi, I'm trying to convert a siamese model (to perform one shot learning) into TensorRT. It's a siamese because there are actually two steps. The features of the first image are extracted by a CNN and then they are used as weights of the kernel of the second CNN for the second image to perform an XCorrelationDepthwise correlation and determine the degree of similiraty between two images. In that particular case I need to pass the output of the first CNN as dynamic Kernel weights of a second CNN as part of the inference graph. Is there a way to achieve that ? I was thinking of recoding by myself the CUDA convolution operation to take two dynamic Tensors arguments.

[bqm1111]

Hi, I'm trying to convert a siamese model (to perform one shot learning) into TensorRT. It's a siamese because there are actually two steps. The features of the first image are extracted by a CNN and then they are used as weights of the kernel of the second CNN for the second image to perform an XCorrelationDepthwise correlation and determine the degree of similiraty between two images. In that particular case I need to pass the output of the first CNN as dynamic Kernel weights of a second CNN as part of the inference graph. Is there a way to achieve that ? I was thinking of recoding by myself the CUDA convolution operation to take two dynamic Tensors arguments.

Do you find any solution to implement Siamese Model in TensorRT. I have the same problem

[bqm1111]

Hi, I'm trying to convert a siamese model (to perform one shot learning) into TensorRT. It's a siamese because there are actually two steps. The features of the first image are extracted by a CNN and then they are used as weights of the kernel of the second CNN for the second image to perform an XCorrelationDepthwise correlation and determine the degree of similiraty between two images. In that particular case I need to pass the output of the first CNN as dynamic Kernel weights of a second CNN as part of the inference graph. Is there a way to achieve that ? I was thinking of recoding by myself the CUDA convolution operation to take two dynamic Tensors arguments.

I guess the Siamese Model you are talking about is SiamRPN network for object tracking. Did you successfully convert it to TensorRT. Could you tell me about its performance?

[ShirGrinblat]

@rmccorm4 @bqm1111 @GabrieldeBlois If you have any solution for converting SiamRPN to TensorRT I would love to hear! I'm trying to convert https://github.com/STVIR/pysot into TensorRT and it doesn't work. At first I converted the model into ONNX successfully,but I'm getting this error when I'm trying to convert ONNX-> TenrsorRT:

• [TensorRT] ERROR: Network must have at least one output
• [TensorRT] ERROR: Network validation failed

Thanks!

[linhnq159]

@rmccorm4 @bqm1111 @GabrieldeBlois If you have any solution for converting SiamRPN to TensorRT I would love to hear! I'm trying to convert https://github.com/STVIR/pysot into TensorRT and it doesn't work. At first I converted the model into ONNX successfully,but I'm getting this error when I'm trying to convert ONNX-> TenrsorRT:

• [TensorRT] ERROR: Network must have at least one output
• [TensorRT] ERROR: Network validation failed

Thanks!

now , did you convert it successfully ? Can you show me cuz i meet the trouble trying to ONNX to TensorRT with F.conv2d on pytorch