search

hunglc007 / tensorflow-yolov4-tflite

YOLOv4, YOLOv4-tiny, YOLOv3, YOLOv3-tiny Implemented in Tensorflow 2.0, Android. Convert YOLO v4 .weights tensorflow, tensorrt and tflite
https://github.com/hunglc007/tensorflow-yolov4-tflite
MIT License
2.23k stars 1.24k forks source link

Cannot reshape array into (1024, 512, 3, 3) with custom yolov4 model #312

Closed thomasgigout closed 3 years ago

thomasgigout commented 3 years ago

Hi there, I'm trying to convert a yolo training into tensorflow model to use it with OpenCV Python. I can convert the default yolov4.weights file into a saved_model.pb, but when I try to run it on a custom yolo model, I got this error : user/tools/tensorflow-yolov4-master/core/utils.py", line 114, in load_weights conv_weights = conv_weights.reshape(conv_shape).transpose([2, 3, 1, 0]) ValueError: cannot reshape array of size 2315379 into shape (1024,512,3,3)

Thanks

pcoder93 commented 3 years ago

Hi,

I am trying to convert darknet custom train model weights to tensorflow.

I am having the exact same issue

covert err

Please advise how to resolve

Thank you.

hhk7734 commented 3 years ago

@pcoder93 can you share your weights and cfg file?

thomasgigout commented 3 years ago

My job was to take a custom model trained by someone before. But I don't know if he used yolov3 or v4. So I tried to make a new custom model using v4 (only with apples and oranges) and I was able to convert it so I think that the previous model wasn't properly trained or something like that.

pcoder93 commented 3 years ago

@hhk7734 Please find the cfg and weight below:

Custom training cfg and weight

hhk7734 commented 3 years ago 
 113  route_12    112,110,108,107,                        ->   13 x  13 x2048
 114  convo_75    512      1 x 1 / 1       13 x  13 x2048 ->   13 x  13 x 512   0.354
 115  convo_76   1024      3 x 3 / 1       13 x  13 x 512 ->   13 x  13 x1024   1.595
 116  convo_77    512      1 x 1 / 1       13 x  13 x1024 ->   13 x  13 x 512   0.177
 117  convo_78    256      1 x 1 / 1       13 x  13 x 512 ->   13 x  13 x 256   0.044
 118  upsam_0                      2x      13 x  13 x 256 ->   26 x  26 x 256  
 119  route_13     85,                                    ->   26 x  26 x 512
 120  convo_79    256      1 x 1 / 1       26 x  26 x 512 ->   26 x  26 x 256   0.177
 121  route_14    120,118,                                ->   26 x  26 x 512
 122  convo_80    256      1 x 1 / 1       26 x  26 x 512 ->   26 x  26 x 256   0.177
 123  convo_81    512      3 x 3 / 1       26 x  26 x 256 ->   26 x  26 x 512   1.595
 124  convo_82    256      1 x 1 / 1       26 x  26 x 512 ->   26 x  26 x 256   0.177
 125  convo_83    512      3 x 3 / 1       26 x  26 x 256 ->   26 x  26 x 512   1.595
 126  convo_84    256      1 x 1 / 1       26 x  26 x 512 ->   26 x  26 x 256   0.177
 127  convo_85    128      1 x 1 / 1       26 x  26 x 256 ->   26 x  26 x 128   0.044
 128  upsam_1                      2x      26 x  26 x 128 ->   52 x  52 x 128  
 129  route_15     54,                                    ->   52 x  52 x 256
 130  convo_86    128      1 x 1 / 1       52 x  52 x 256 ->   52 x  52 x 128   0.177
 131  route_16    130,128,                                ->   52 x  52 x 256
 132  convo_87    128      1 x 1 / 1       52 x  52 x 256 ->   52 x  52 x 128   0.177
 133  convo_88    256      3 x 3 / 1       52 x  52 x 128 ->   52 x  52 x 256   1.595
 134  convo_89    128      1 x 1 / 1       52 x  52 x 256 ->   52 x  52 x 128   0.177
 135  convo_90    256      3 x 3 / 1       52 x  52 x 128 ->   52 x  52 x 256   1.595
 136  convo_91    128      1 x 1 / 1       52 x  52 x 256 ->   52 x  52 x 128   0.177
 137  convo_92    256      3 x 3 / 1       52 x  52 x 128 ->   52 x  52 x 256   1.595
 138  convo_93     18      1 x 1 / 1       52 x  52 x 256 ->   52 x  52 x  18   0.025
 139  yolo _0    iou_loss: ciou, iou_norm: 0.07, obj_norm: 1.0, cls_norm: 1.0, 
                 scale_x_y: 1.2
 140  route_17    136,                                    ->   52 x  52 x 128
 141  convo_94    512      3 x 3 / 1       52 x  52 x 128 ->   52 x  52 x 512   3.190
 142  convo_95    256      1 x 1 / 1       52 x  52 x 512 ->   52 x  52 x 256   0.709
 143  convo_96    512      3 x 3 / 1       52 x  52 x 256 ->   52 x  52 x 512   6.380
 144  convo_97    256      1 x 1 / 1       52 x  52 x 512 ->   52 x  52 x 256   0.709
 145  convo_98    512      3 x 3 / 1       52 x  52 x 256 ->   52 x  52 x 512   6.380
 146  convo_99     18      1 x 1 / 1       52 x  52 x 512 ->   52 x  52 x  18   0.050
 147  yolo _1    iou_loss: ciou, iou_norm: 0.07, obj_norm: 1.0, cls_norm: 1.0, 
                 scale_x_y: 1.1
 148  route_18    144,                                    ->   52 x  52 x 256
 149  convo_100   512      3 x 3 / 2       52 x  52 x 256 ->   26 x  26 x 512   1.595
 150  route_19    149,113,                                ->   26 x  26 x2560
 151  convo_101   512      1 x 1 / 1       26 x  26 x2560 ->   26 x  26 x 512   1.772
 152  convo_102  1024      3 x 3 / 1       26 x  26 x 512 ->   26 x  26 x1024   6.380
 153  convo_103   512      1 x 1 / 1       26 x  26 x1024 ->   26 x  26 x 512   0.709
 154  convo_104  1024      3 x 3 / 1       26 x  26 x 512 ->   26 x  26 x1024   6.380
 155  convo_105   512      1 x 1 / 1       26 x  26 x1024 ->   26 x  26 x 512   0.709
 156  convo_106  1024      3 x 3 / 1       26 x  26 x 512 ->   26 x  26 x1024   6.380
 157  convo_107    18      1 x 1 / 1       26 x  26 x1024 ->   26 x  26 x  18   0.025
 158  yolo _2    iou_loss: ciou, iou_norm: 0.07, obj_norm: 1.0, cls_norm: 1.0, 
                 scale_x_y: 1.05
Total B(1e9)flops: 89.479

your cfg file seems to be weird.

see line 113 and line 150 113's output shape is 13x13x2048 149's output shape is 26x26x512

150 route_19 concatenate two output. but it is impossible. And, to detect small object more, your cfg removes some layer. this repo does not support layer transformation. (I guess) you can change only the items related to the part written in the script below. https://github.com/hunglc007/tensorflow-yolov4-tflite/blob/9f16748aa3/core/config.py

and darknet is c code. even if array shape is different, in c code, are just miscalculated. there are no restrictions on sahpe. Maybe that's why it worked on darknet.

jaymattkr commented 3 years ago

save_model.py uses coco.names to get NUM_CLASS by default

you need to define your custom classes. See core/config.py

yingtina commented 3 years ago

@thomasgigout Have you solved this problem and how? I have the same problem. Because I need to detect small objects, I changed filters, stride input_size ... in the config file in Darknet. I hope you can give me some hit.

thomasgigout commented 3 years ago

@thomasgigout Have you solved this problem and how? I have the same problem. Because I need to detect small objects, I changed filters, stride input_size ... in the config file in Darknet. I hope you can give me some hit.

Hi @yingtina, as I said here (https://github.com/hunglc007/tensorflow-yolov4-tflite/issues/312#issuecomment-781955906) the pre trained model I used was probably the problem, because I tried to use another model, trained by myself this time, and I was able to convert it. So maybe you should give attention to the version of yolo before converting it. However, I kinda gave up this conversion and stayed on yolov4