python3: yolo_layer.cu:121 Error when converting custom yolov4 model from onnx_to_tenssort

[seabass1217]

Hi all,

I'm receiving the below error when try to convert my mode to tensorrt. I have the error log for addition information. Any help is greatly appreciated.

python3: yolo_layer.cu:121: virtual nvinfer1::Dims nvinfer1::YoloLayerPlugin::getOutputDimensions(int, const Dims*, int): Assertion `inputs[0].d[1] == mYoloHeight' failed. Aborted (core dumped) onnx_to_trt_log.txt

[jkjung-avt]

It looked like you're using "stride=4" in your custom yolo model. Please modify the source code accordingly.

https://github.com/jkjung-avt/tensorrt_demos/blob/1f8773b3f6091d2e3441c0d6ce5714e332421f8d/yolo/plugins.py#L93-L96

To be modified as:

     yolo_whs = [ 
         [w // 32, h // 32], [w // 16, h // 16], 
         [w //  4, h //  4]] 

[seabass1217]

@jkjung-avt, Thanks! I will make the modification.

[seabass1217]

@jkjung-avt, I modified the plugins.py file as you suggested and now I'm getting another error:

python3: yolo_layer.cu:355: virtual nvinfer1::IPluginV2IOExt nvinfer1::YoloPluginCreator::createPlugin(const char, const nvinfer1::PluginFieldCollection*): Assertion `input_multiplier == 64 || input_multiplier == 32 || input_multiplier == 16 || input_multiplier == 8' failed.

[jkjung-avt]

Ah, you need to modify source code of the plugin and re-compile it as well.

https://github.com/jkjung-avt/tensorrt_demos/blob/67c4eb0f5e8a46656eb65073e5685dc7654498d8/plugins/yolo_layer.cu#L354-L355

To be modified as:

 assert(input_multiplier == 64 || input_multiplier == 32 || \ 
        input_multiplier == 16 || input_multiplier == 8 || input_multiplier == 4); 

[seabass1217]

@jkjung-avt,

Thanks! I had just figured it out... also had to remember to do 'make' to make mods to .cu file.