YOLOv5 3.0 Assertion failed with custom anchors

[moromatt]

Hi guys, I'm trying to run the sample_detector.cpp with my trained model. As far as I can tell the weights are loaded correctly, instead the cfg file seems to have some trouble with the Detect layer. I've tried to convert and thed run a pre-trained model of yolov5l and it works. In my case, the number of classes are 6, height and width are both 800. They are correctly set inside the cfg file.

When I try to run the sample_detector.cpp it returns this.

File does not exist : ../configs/yolov5-3.0/yolov5l_custom-kFLOAT-batch1.engine
Loading pre-trained weights...
Loading complete!
      layer               inp_size            out_size
(1)   Focus             3 x 800 x 800      64 x 400 x 400
(2)   Conv             64 x 400 x 400     128 x 200 x 200
(3)   BottleneckCSP   128 x 200 x 200     128 x 200 x 200
(4)   Conv            128 x 200 x 200     256 x 100 x 100
(5)   BottleneckCSP   256 x 100 x 100     256 x 100 x 100
(6)   Conv            256 x 100 x 100     512 x  50 x  50
(7)   BottleneckCSP   512 x  50 x  50     512 x  50 x  50
(8)   Conv            512 x  50 x  50    1024 x  25 x  25
(9)   SPP            1024 x  25 x  25    1024 x  25 x  25
(10)  BottleneckCSP  1024 x  25 x  25    1024 x  25 x  25
(11)  Conv           1024 x  25 x  25     512 x  25 x  25
(12)  Upsample        512 x  25 x  25     512 x  50 x  50
(13)  Concat          512 x  50 x  50    1024 x  50 x  50
(14)  BottleneckCSP  1024 x  50 x  50     512 x  50 x  50
(15)  Conv            512 x  50 x  50     256 x  50 x  50
(16)  Upsample        256 x  50 x  50     256 x 100 x 100
(17)  Concat          256 x 100 x 100     512 x 100 x 100
(18)  BottleneckCSP   512 x 100 x 100     256 x 100 x 100
(19)  Conv            256 x 100 x 100     256 x  50 x  50
(20)  Concat          256 x  50 x  50     512 x  50 x  50
(21)  BottleneckCSP   512 x  50 x  50     512 x  50 x  50
(22)  Conv            512 x  50 x  50     512 x  25 x  25
(23)  Concat          512 x  25 x  25    1024 x  25 x  25
(24)  BottleneckCSP  1024 x  25 x  25    1024 x  25 x  25
Assertion failed: size == (map_wts_[s_layer_name_ + ".weight"].size()), file c:\users\moro\desktop\yolo_tensorrt\modules\trt_utils.cpp, line 854

My .cfg file is this one:

[net]
width=800
height=800
channels=3
batch=1
nc=6
depth_multiple=1.0
width_multiple=1.0
anchors=23, 24, 27, 34, 46, 32, 229, 66, 97, 29, 34, 146, 75, 88, 193, 40, 64, 210, 121, 129, 233, 161, 183, 248, 133, 556, 407, 234, 177, 765, 785, 269, 508, 599, 784, 813

[Focus]
from=-1
number=1
args=64, 3

[Conv]
from=-1
number=1
args=128, 3, 2

[BottleneckCSP]
from=-1
number=3
args=128

[Conv]
from=-1
number=1
args=256, 3, 2

[BottleneckCSP]
from=-1
number=9
args=256

[Conv]
from=-1
number=1
args=512, 3, 2

[BottleneckCSP]
from=-1
number=9
args=512

[Conv]
from=-1
number=1
args=1024, 3, 2

[SPP]
from=-1
number=1
args=1024, [5, 9, 13]

[BottleneckCSP]
from=-1
number=3
args=1024, False

[Conv]
from=-1
number=1
args=512, 1, 1

[nn.Upsample]
from=-1
number=1
args='None', 2, 'nearest'

[Concat]
from=-1, 6
number=1
args=1

[BottleneckCSP]
from=-1
number=3
args=512, False

[Conv]
from=-1
number=1
args=256, 1, 1

[nn.Upsample]
from=-1
number=1
args='None', 2, 'nearest'

[Concat]
from=-1, 4
number=1
args=1

[BottleneckCSP]
from=-1
number=3
args=256, False

[Conv]
from=-1
number=1
args=256, 3, 2

[Concat]
from=-1, 14
number=1
args=1

[BottleneckCSP]
from=-1
number=3
args=512, False

[Conv]
from=-1
number=1
args=512, 3, 2

[Concat]
from=-1, 10
number=1
args=1

[BottleneckCSP]
from=-1
number=3
args=1024, False

[Detect]
from=17, 20, 23
number=1
args='nc', 'anchors'

The number of anchors are more than the pre-trained cfg file, they are generated with the YOLOv5 3.0 autoanchors and they works just fine when the model is loaded for the inference with PyTorch.

Can you guys please help me? Thanks in advance!

[enazoe]

the cfg file and weigths file are not matched

[moromatt]

Ok thanks @enazoe, but honestly I don't understand the reason. As you can see from the yolo_custom.cfg file: width, height and number of classes are correctly set. Furthermore the sample_detector.cpp output shows the same:

      layer               inp_size            out_size
(1)   Focus             3 x 800 x 800      64 x 400 x 400

My original config yaml file is this:

# parameters
nc: 6  # number of classes
depth_multiple: 1.0  # model depth multiple
width_multiple: 1.0  # layer channel multiple

# anchors
anchors:  
    - [23,24,  27,34,  46,32] 
    - [229,66,  97,29,  34,146] 
    - [75,88,  193,40,  64,210]
    - [121,129,  233,161,  183,248]
    - [133,556,  407,234,  177,765]
    - [785,269,  508,599,  784,813]

# YOLOv5 backbone
backbone:
  # [from, number, module, args]
  [[-1, 1, Focus, [64, 3]],  # 0-P1/2
   [-1, 1, Conv, [128, 3, 2]],  # 1-P2/4
   [-1, 3, BottleneckCSP, [128]],
   [-1, 1, Conv, [256, 3, 2]],  # 3-P3/8
   [-1, 9, BottleneckCSP, [256]],
   [-1, 1, Conv, [512, 3, 2]],  # 5-P4/16
   [-1, 9, BottleneckCSP, [512]],
   [-1, 1, Conv, [1024, 3, 2]],  # 7-P5/32
   [-1, 1, SPP, [1024, [5, 9, 13]]],
   [-1, 3, BottleneckCSP, [1024, False]],  # 9
  ]

# YOLOv5 head
head:
  [[-1, 1, Conv, [512, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 6], 1, Concat, [1]],  # cat backbone P4
   [-1, 3, BottleneckCSP, [512, False]],  # 13

   [-1, 1, Conv, [256, 1, 1]],
   [-1, 1, nn.Upsample, [None, 2, 'nearest']],
   [[-1, 4], 1, Concat, [1]],  # cat backbone P3
   [-1, 3, BottleneckCSP, [256, False]],  # 17 (P3/8-small)

   [-1, 1, Conv, [256, 3, 2]],
   [[-1, 14], 1, Concat, [1]],  # cat head P4
   [-1, 3, BottleneckCSP, [512, False]],  # 20 (P4/16-medium)

   [-1, 1, Conv, [512, 3, 2]],
   [[-1, 10], 1, Concat, [1]],  # cat head P5
   [-1, 3, BottleneckCSP, [1024, False]],  # 23 (P5/32-large)

   [[17, 20, 23], 1, Detect, [nc, anchors]],  # Detect(P3, P4, P5)
  ]

What am I missing? Thanks in advance!

[moromatt]

Hi @enazoe, I'm still looking for solutions, but with no fortune, could you please give me any hints in order to find out what I'm doing wrong? Thanks in advance!

[enazoe]

if you use yolov5-3.0 ，you should checkout to 322a1310adf28fb911f1e10415aa5de6432126aa

[moromatt]

Hi @enazoe , yes I'm actually using yolov5-3.0. Finally I successfully loaded a model trained with my custom data and the relative number of classes, but I had to keep the same number of anchors like the pre-trained model. Honestly, I do not understand what I should check in 322a131.

I assume that I should change something inside the cfg file, in order to match the weigths file with the cfg file in order to have more anchors. Can you suggest what I should try to modify in my cfg file?

Thanks again for the help, I really appreciate that.