Open LFRS opened 3 years ago
I am having the same issue. After first few hundred iteration my average loss stays at around 37, 42.
I am having the same issue. After first few hundred iteration my average loss stays at around 37, 42.
checkout an old commit it solved my problem.
i also meet the same issue. and i don't know why?
I am having the same issue. After first few hundred iteration my average loss stays at around 37, 42.
checkout an old commit it solved my problem.
What commit was it?
@ROBYER1 ,
I am having the same issue. After first few hundred iteration my average loss stays at around 37, 42.
checkout an old commit it solved my problem.
What commit was it?
Maybe this one: https://github.com/AlexeyAB/darknet/issues/7097#issuecomment-742737350 (not sure but looks like)
Hi,
I'm having issues training a custom dataset, i'm getting "high Avg Loss" and i don't know why.
The dataset has 1 Class with:
The "yolov4-obj.cfg" file is attached but the changes for one class were:
The results that i'm getting are this, "High Avg Loss" and "High mAP":
What i'm doing wrong??? Thanks.
------ "yolov4-obj.cfg" ------
[net]
Testing
batch=1
subdivisions=1
Training
batch=16 subdivisions=16 width=160 height=160 channels=3 momentum=0.949 decay=0.0005 angle=0 saturation = 1.5 exposure = 1.5 hue=.1
learning_rate=0.001 burn_in=1000 max_batches=6000 policy=steps steps=48000,54000 scales=.1,.1
cutmix=1
mosaic=1
:104x104 54:52x52 85:26x26 104:13x13 for 416
[convolutional] batch_normalize=1 filters=32 size=3 stride=1 pad=1 activation=mish
Downsample
[convolutional] batch_normalize=1 filters=64 size=3 stride=2 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[route] layers = -2
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=32 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[route] layers = -1,-7
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
Downsample
[convolutional] batch_normalize=1 filters=128 size=3 stride=2 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[route] layers = -2
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=64 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=64 size=1 stride=1 pad=1 activation=mish
[route] layers = -1,-10
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
Downsample
[convolutional] batch_normalize=1 filters=256 size=3 stride=2 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[route] layers = -2
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=mish
[route] layers = -1,-28
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
Downsample
[convolutional] batch_normalize=1 filters=512 size=3 stride=2 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[route] layers = -2
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=mish
[route] layers = -1,-28
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
Downsample
[convolutional] batch_normalize=1 filters=1024 size=3 stride=2 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[route] layers = -2
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=mish
[shortcut] from=-3 activation=linear
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=mish
[route] layers = -1,-16
[convolutional] batch_normalize=1 filters=1024 size=1 stride=1 pad=1 activation=mish stopbackward=800
##########################
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=1024 activation=leaky
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
SPP
[maxpool] stride=1 size=5
[route] layers=-2
[maxpool] stride=1 size=9
[route] layers=-4
[maxpool] stride=1 size=13
[route] layers=-1,-3,-5,-6
End SPP
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=1024 activation=leaky
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[upsample] stride=2
[route] layers = 85
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[route] layers = -1, -3
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=512 activation=leaky
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=512 activation=leaky
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky
[upsample] stride=2
[route] layers = 54
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky
[route] layers = -1, -3
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=256 activation=leaky
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=256 activation=leaky
[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky
##########################
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=256 activation=leaky
[convolutional] size=1 stride=1 pad=1 filters=18 activation=linear
[yolo] mask = 0,1,2 anchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401 classes=1 num=9 jitter=.3 ignore_thresh = .7 truth_thresh = 1 scale_x_y = 1.2 iou_thresh=0.213 cls_normalizer=1.0 iou_normalizer=0.07 iou_loss=ciou nms_kind=greedynms beta_nms=0.6 max_delta=5
[route] layers = -4
[convolutional] batch_normalize=1 size=3 stride=2 pad=1 filters=256 activation=leaky
[route] layers = -1, -16
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=512 activation=leaky
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=512 activation=leaky
[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=512 activation=leaky
[convolutional] size=1 stride=1 pad=1 filters=18 activation=linear
[yolo] mask = 3,4,5 anchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401 classes=1 num=9 jitter=.3 ignore_thresh = .7 truth_thresh = 1 scale_x_y = 1.1 iou_thresh=0.213 cls_normalizer=1.0 iou_normalizer=0.07 iou_loss=ciou nms_kind=greedynms beta_nms=0.6 max_delta=5
[route] layers = -4
[convolutional] batch_normalize=1 size=3 stride=2 pad=1 filters=512 activation=leaky
[route] layers = -1, -37
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=1024 activation=leaky
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=1024 activation=leaky
[convolutional] batch_normalize=1 filters=512 size=1 stride=1 pad=1 activation=leaky
[convolutional] batch_normalize=1 size=3 stride=1 pad=1 filters=1024 activation=leaky
[convolutional] size=1 stride=1 pad=1 filters=18 activation=linear
[yolo] mask = 6,7,8 anchors = 12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401 classes=1 num=9 jitter=.3 ignore_thresh = .7 truth_thresh = 1 random=1 scale_x_y = 1.05 iou_thresh=0.213 cls_normalizer=1.0 iou_normalizer=0.07 iou_loss=ciou nms_kind=greedynms beta_nms=0.6 max_delta=5