ValueError: cannot reshape array of size 3132 into shape (7)

[Darshcg]

Hi, @jkjung-avt,

I am using https://github.com/jkjung-avt/tensorrt_demos/blob/master/trt_yolo.py script(with some modifications) for my Custom Model for an Inference to run on Jetson Nano.

After running inference on one image, I am getting a trt_outputs(https://github.com/jkjung-avt/tensorrt_demos/blob/e039e0824876d46443aa19e9ad7cf8f7723c713e/utils/yolo_with_plugins.py#L279) as: trt_outputs=== [array([ -4.1128, -4.0136, 12.006, ..., -150.05, 350.51, 374.29], dtype=float32), array([ -0.68137, -0.70507, -0.65604, ..., -0.71709, -0.72069, -0.66634], dtype=float32), array([ 3.9349, 3.9753, 4.1016, ..., 104.41, 102.56, 103.08], dtype=float32)]

But when I am passing the trt_outputs for Post Process function(https://github.com/jkjung-avt/tensorrt_demos/blob/e039e0824876d46443aa19e9ad7cf8f7723c713e/utils/yolo_with_plugins.py#L83), I am getting an error as: detections = np.concatenate([o.reshape(-1, 7) for o in trt_outputs], axis=0) ValueError: cannot reshape array of size 3132 into shape (7)

Can you please guide me to get rid of this issue?

Thanks, Darshan

[jkjung-avt]

Your output shapes are not correct. It is most likely because you do not mark the correct layers as outputs. You need to find out the names of the convolution layers **right before the "yolo" layers and modify "yolo_to_onnx.py" accordingly.

Please refer to the following lines of code. (The numbers 139, 150, 161 would likely need to be modified for your custom model.)

https://github.com/jkjung-avt/tensorrt_demos/blob/82dbbacc86cf558b1f70fca938f0e13130b281f2/yolo/yolo_to_onnx.py#L922-L924

[jkjung-avt]

@Darshcg Has this been resolved?

[Darshcg]

Hi @jkjung-avt, Thanks a lot for your consistent support. I feel that the trt_outputs we are getting is not from the correct Output Layer. What I feel is I am not doing the conversion properly. I am attaching all the details of the Model and the Scripts for the conversion below.

@jkjung-avt My Goal is: We have a Face Recognition model trained on our custom datasets. That Face Recognition has to be converted to the TRT model and run on the Jetson Nano. For Face Detection: we used Retinaface model(Resnet50 as a backend). For Face Recognition: we used IR-SE50(https://github.com/deepinsight/insightface). Both are PyTorch-based.

Now I am working on Face Detection. The script I used for pt to ONNX is attached below:

from models import *
from datasets import letterbox #*
from utils import *
import numpy as np
import sys, random
import torch
import torchvision
from torchvision import models, transforms
from PIL import Image
import io
import torch.nn as nn
from torch.autograd import Variable
import cv2
import os
import logging

img_size=(416, 416)
cfg = 'yolov3-spp.cfg'
imgsz = img_size

import torch
import torch.onnx
model_pt_path = "last.pt"
model_def_path = "models.py"
model = Darknet(cfg, imgsz)
model.load_state_dict(torch.load(model_pt_path),strict=False)
dummy_input = torch.randn(1,3,416,416)
torch.onnx.export(model, dummy_input, "onnx_model_name.onnx")

And the code I used for ONNX to TRT conversion is:

import tensorrt as trt
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt_runtime = trt.Runtime(TRT_LOGGER)
def build_engine(onnx_path, shape = [1,1,224,224]):
    with trt.Builder(TRT_LOGGER) as builder, builder.create_network(1) as network, trt.OnnxParser(network, TRT_LOGGER) as parser:
        builder.max_workspace_size = (1<< 30)
        with open(onnx_path, 'rb') as model:
            parser.parse(model.read())
        network.get_input(0).shape = shape
        engine = builder.build_cuda_engine(network)
        return engine
def save_engine(engine, file_name):
    buf = engine.serialize()def load_engine(trt_runtime, plan_path):
    with open(engine_path, 'rb') as f:
        engine_data = f.read()
def load_engine(trt_runtime, plan_path):
    with open(engine_path, 'rb') as f:
        engine_data = f.read()
    engine = trt_runtime.deserialize_cuda_engine(engine_data)
    return engine
from onnx import ModelProto

engine_name = "PMH.plan"
onnx_path = "PMH.onnx"

model = ModelProto()
with open(onnx_path, "rb") as f:
    model.ParseFromString(f.read()) 
batch_size = 1 
d0 = model.graph.input[0].type.tensor_type.shape.dim[1].dim_value
d1 = model.graph.input[0].type.tensor_type.shape.dim[2].dim_value
d2 = model.graph.input[0].type.tensor_type.shape.dim[3].dim_value
shape = [batch_size , d0, d1 ,d2]

engine = build_engine(onnx_path)
save_engine(engine, engine_name) 

These are the scripts I am using which are standard and has been given in the Oficial Pytorch and Nvidia Website. I am using as it is for our models(with the little changes). So is there any changes has to be made in the scripts in the scripts Ihave attached above?

Would be a great help from your side.

Regards, Darshan

[jkjung-avt]

I have only tried to convert darknet models to TensorRT. I'm afraid I don't have time to help you on this...

[Darshcg]

Hi, @jkjung-avt,

It is completely okay. I am getting progress in resolving an issue. Thanks for your consistent support.

Btw, is your repo supports for **YOLOv3-SPP-ultralytics*** model(https://github.com/ultralytics/yolov3)? If not what are the changes to be made?

Thank you.

[jkjung-avt]

I have not studied the YOLOv3-SPP-ultralytics model before. Since it is based on pytorch, I think quite some modifications of yolo_to_onnx.py and onnx_to_tensorrt.py would be required.

[Darshcg]

Thanks a lot, @jkjung-avt for your help.

[Darshcg]

I have not studied the YOLOv3-SPP-ultralytics model before. Since it is based on pytorch, I think quite some modifications of yolo_to_onnx.py and onnx_to_tensorrt.py would be required.

Hi @jkjung-avt, Your tensorrt_demos repo also supports the YOLOv3-SPP-Ultralytics model. I have tested it.

Thank you.

[jkjung-avt]

Great. Thanks for sharing this.

[leeyunhome]

Hi, @jkjung-avt,

It is completely okay. I am getting progress in resolving an issue. Thanks for your consistent support.

Btw, is your repo supports for **YOLOv3-SPP-ultralytics*** model(https://github.com/ultralytics/yolov3)? If not what are the changes to be made?

Thank you.

Hi, @Darshcg

I'm also using yolov3 (https://github.com/ultralytics/yolov3). I am also using jetson nano.

What and how you changed it Or, can you share the changed code through the project? I used yolov3 to recognize the license plate.

Thank you.

[Darshcg]

Hi @leeyunhome,

Sorry for the late reply(I saw this today). if you are using yolov3 for your project, then just follow https://github.com/jkjung-avt/tensorrt_demos#demo-5-yolov4 and refer this:https://jkjung-avt.github.io/trt-yolov3-custom/ for conversion and inference, it is just about passing the right arguments for the conversion and Inference.

Thanks