TypeError: buffer is too small for requested array

[ryotaro137]

I tried to convert orignal model to ONNX models. But, the eroor occurs when I executed !python yolo_to_onnx.py -c 4 --model yolov5-custom

error message

Building ONNX graph...
Traceback (most recent call last)  File "yolo_to_onnx.py", line 1065, in <module>
    main()
  File "yolo_to_onnx.py", line 1053, in main
    verbose=True)
  File "yolo_to_onnx.py", line 585, in build_onnx_graph
    params)
  File "yolo_to_onnx.py", line 459, in load_conv_weights
    conv_params, 'conv', 'weights')
  File "yolo_to_onnx.py", line 487, in _create_param_tensors
    conv_params, param_category, suffix)
  File "yolo_to_onnx.py", line 517, in _load_one_param_type
    buffer=self.weights_file.read(param_size * 4))
TypeError: buffer is too small for requested array

I have verified that cfg and weights are compatible. Even if I look at the past issues, I don't know how to deal with it, so I asked a question.

I would be grateful if you could let me know if there is any other reason.

Thank you in advance.

[jkjung-avt]

TypeError: buffer is too small for requested array

This means the "yolo_to_onnx.py" code failed to parse and load the cfg & weights file. It's likely due to some parameters (which affects weight sizes) in certain layers which are not implemented in "yolo_to_onnx.py".

[ryotaro137]

Perhaps it is because I wrote padding -> pad in cfg. I will do some more research.

[jkjung-avt]

Yes, it could be due to padding/pad settings in convolutional layers. Refer to the source code below. "yolo_to_onnx.py" assumes all convolutional layers are using "SAME_LOWER" padding. It does not reference the actual padding/pad parameter in the cfg file at all...

https://github.com/jkjung-avt/tensorrt_demos/blob/b757be60af727adf50690fc19209e63d366d8e51/yolo/yolo_to_onnx.py#L683-L723

[ryotaro137]

I replaced SAME_LOWER into NOTSET in code. But same erroe occured. So I replaced pad into padding in cfg and train from scratch. After training, I will tell results as soon as possible. Thank you in advance.

[ryotaro137]

My model has finished training and I tried to convert onnx model. But, same error occuerd. I checked all conv paremeter pad=1 in cfg.

Would you teach me other causes? Thank you in advance.

Traceback (most recent call last):
  File "yolo_to_onnx.py", line 1065, in <module>
    main()
  File "yolo_to_onnx.py", line 1053, in main
    verbose=True)
  File "yolo_to_onnx.py", line 585, in build_onnx_graph
    params)
  File "yolo_to_onnx.py", line 459, in load_conv_weights
    conv_params, 'conv', 'weights')
  File "yolo_to_onnx.py", line 487, in _create_param_tensors
    conv_params, param_category, suffix)
  File "yolo_to_onnx.py", line 517, in _load_one_param_type
    buffer=self.weights_file.read(param_size * 4))
TypeError: buffer is too small for requested array

[jkjung-avt]

I checked all conv paremeter pad=1 in cfg.

The above is not correct.

• For convolutional layer with size=1, pad should be 0.
• For convolutional layer with size=3, pad should be 1.
• For convolutional layer with size=5, pad should be 2.
• For convolutional layer with size=7, pad should be 3.
• ......

In other words, the "SAME_LOWER" padding would produce the same output dimension if stride is 1.

[ryotaro137]

I found pad in darknet cfg.(https://github.com/AlexeyAB/darknet/wiki/CFG-Parameters-in-the-different-layers)

pad=1 - if 1 will be used padding = size/2, if 0 the will be used parameter padding= (0 by default)

My model used size=3 or 1, so setting pad=1 means no probelm.

Also, I found error conv in my cfg.

error

01_convolutional
002_convolutional
003_convolutional
・・・・・・・・・・・
067_convolutional
069_convolutional
TypeError: buffer is too small for requested array

069_convolutional is below

[convolutional]
batch_normalize=1
filters=64
size=1
stride=1
pad=1
activation=swish

This means that other conv parameters is correct, don't you?

[ryotaro137]

Maybe, I think DepthwiseConv cannot correspond to onnx model. If it is true, please teach me how to convert groupconv block to onnx. Thank you in advance.

[jkjung-avt]

Maybe, I think DepthwiseConv cannot correspond to onnx model.

The "yolo_to_onnx.py" code indeed does not handle DepthwiseConv at all. Such layers would be converted to normal Conv, which consumes more weights values than DepthwiseConv. So it's no surprise that you encounter "TypeError: buffer is too small for requested array" when you have DepthwiseConv in the cfg file.

[ryotaro137]

The "yolo_to_onnx.py" code indeed does not handle DepthwiseConv at all. .

I can understand the cause of error. If you have a time, please teach me how to convert DepthwiseConv to onnx model. Thank you in advance.