Gaussian YOLOv3 (+3.1% mAP@0.5...0.95 on COCO) , (+3.0% mAP@0.7 on KITTI) , (+3.5% mAP@0.75 on BDD)

[phucnhs]

Have you tried the Gaussian object detection method?

• cfg files: https://github.com/jwchoi384/Gaussian_YOLOv3/tree/master/cfg
• BDD100k weights file: https://drive.google.com/open?id=1Eutnens-3z6o4LYe0PZXJ1VYNwcZ6-2Y
• paper: https://arxiv.org/abs/1904.04620v2
• source code: https://github.com/jwchoi384/Gaussian_YOLOv3

[AlexeyAB]

What do you mean? Can you provide URL to article/paper?

[phucnhs]

This is object detection by new method this is paper : https://arxiv.org/pdf/1904.04620.pdf Here is the code, you can refer to it because I am not confident about this code : https://github.com/jwchoi384/Gaussian_YOLOv3

[lsd1994]

Furthermore, on the COCO dataset [14], the AP of Gaussian YOLOv3 is 36.1, which is 3.1 higher than YOLOv3. In particular, the AP75 (i.e., strict metric) of Gaussian YOLOv3 is 39.0, which is 4.6 higher than that of YOLOv3.

It seems Gaussian YOLOv3 is better than YOLOv3 on COCO dataset, especially in strict metric.

[AlexeyAB]

So

• Yolov3 default 608x608 - 33.0% AP@0.5...0.95 and 34.4% AP@0.75 https://arxiv.org/pdf/1804.02767v1.pdf

• Yolov3 Gaussian 608x608 - 36.1% AP@0.5...0.95 and 39.0% AP@0.75 https://arxiv.org/pdf/1904.04620v2.pdf

---

https://arxiv.org/pdf/1902.09630v2.pdf

• Yolov3 default MSE 608x608 - 31.4% AP@0.5...0.95 and 33.3% AP@0.75 - testdev 2018
• Yolov3 GIoU 608x608- 33.3% AP@0.5...0.95 and 36.2% AP@0.75 - testdev 2018
• Yolov3 GIoU 608x608- 33.5% AP@0.5...0.95 and 35.9% AP@0.75 - val 2014

GIoU: https://github.com/AlexeyAB/darknet/issues/3249

[sctrueew]

@AlexeyAB Hi,

Have you tried to combine Gaussian and GIoU? If yes, Could you share the result?

Thanks

[tuteming]

where I can get the Gaussian YOLOv3 config file?

[AlexeyAB]

@tuteming

cfg files: https://github.com/jwchoi384/Gaussian_YOLOv3/tree/master/cfg BDD100k weights file: https://drive.google.com/open?id=1Eutnens-3z6o4LYe0PZXJ1VYNwcZ6-2Y

[AlexeyAB]

I added yolo_v3_tiny_pan3 matrix_gaussian aa_ae_mixup.cfg.txt model.

More: https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-494148968

---

Model (cfg & weights) network size = 544x544	Training chart	Validation video	BFlops	Inference time RTX2070, ms	mAP, %
yolo_v3_tiny_pan3 matrix_gaussian aa_ae_mixup.cfg.txt and weights-file		video	13	19.0 ms	57.2%

[Kyuuki93]

@AlexeyAB Hi, I trained yolov3-spp.cfg and yolov3-spp-gs.cfg in my Custom Data, the yolov3-spp-gs.cfg just replaced [yolo] to [Gaussian yolo]. My data was one-class dataset with 130k images, but for some reason there no validation data, so I can't report the mAP value. There are some observation:

1. Gaussian yolo got high avgloss in the train,
2. Network with Gaussian yolo layer get lower confidence of object,
3. Network with Gaussian yolo layer decrease the wrong detected boxes but aggravated blinking issue in video

I think maybe in the test, * (1.0 - uc_aver)in line 454 of gaussian_yolo_layer.c was not necessary .

Three some video results, extract code :xqxu all results in the -thresh 0.5,
spp.mp4 was the result of yolov3-spp.cfg spp-gs.mp4 was the result of yolov3-spp-gs.cfg spp-gs2.mp4 was the result of yolov3-spp-gs.cfg and removed * (1.0 - uc_aver)in [gaussian_yolo_layer.c]

[AlexeyAB]

My data was one-class dataset with 130k images, but for some reason there no validation data, so I can't report the mAP value.

Just set valid=train.txt in obj.data file

Gaussian yolo got high avgloss in the train, Network with Gaussian yolo layer get lower confidence of object, Network with Gaussian yolo layer decrease the wrong detected boxes but aggravated blinking issue in video

You should use lower -threshold 0.2 or 0.15` for [Gaussian yolo] instead of default 0.24

I think maybe in the test, * (1.0 - uc_aver)in line 454 of gaussian_yolo_layer.c was not necessary .

It is necessary if you want to have high AP@0.5...0.95

[AlexeyAB]

@zpmmehrdad

I added GIoU to the [Gaussian_yolo] layer.

So now you can use for training:

[Gaussian_yolo]
mask = 0,1,2
anchors = 7,10, 14,24, 27,43, 32,97, 57,64, 92,109, 73,175, 141,178, 144,291
classes=10
num=9
jitter=.3
ignore_thresh = .5
truth_thresh = 1
iou_thresh=0.213
uc_normalizer=1.0
cls_normalizer=1.0
iou_normalizer=0.5
iou_loss=giou
scale_x_y=1.1
random=1

[yrc08]

@AlexeyAB Hi! I want to use the Gaussian_yolov3_BDD.cfg training to use darknet53.conv.74 or Gaussian_yolov3_BDD.weights.

[AlexeyAB]

@yrc08 Use darknet53.conv.74 for training and Gaussian_yolov3_BDD.weights for detection.

[yrc08]

@AlexeyAB Hi! In other words, I want to use darknet53.conv.74 when I want to train my own data set. Is my understanding correct?

[LukeAI]

Just to share, I found the gaussian yolo gave me a small boost in mAP, around that claimed in the paper (which is unusual in my experience!)

Gaussian YOLO

Yolov3_spp_swish_scale

I'm currently trying out Gaussian_yolo_spp_swish...

[AlexeyAB]

I'm currently trying out Gaussian_yolo_spp_swish...

Also try then to add params to each of [Gaussian_yolo] layer and train. It requires the latest version of Darknet

iou_thresh=0.3
scale_x_y=1.1

So compare Gaussian_yolo_spp_swish... vs Gaussian_yolo_spp_swish_scale_iou|_thresh...

[LukeAI]

I'm currently trying out Gaussian_yolo_spp_swish...

Also try then to add params to each of [Gaussian_yolo] layer and train. It requires the latest version of Darknet

iou_thresh=0.3
scale_x_y=1.1

So compare Gaussian_yolo_spp_swish... vs Gaussian_yolo_spp_swish_scale_iou|_thresh...

So add the same to every gaussian_yolo? As opposed to scale_x_y=1.05 , 1.1, 1.2 in the different layers?

[LukeAI]

what is the iou_thresh?

[AlexeyAB]

As opposed to scale_x_y=1.05 , 1.1, 1.2 in the different layers?

Or you can add different values for different yolo-layers: scale_x_y=1.05 (for 17x17) , 1.1 (34x34) 1.2 (68x68).

what is the iou_thresh?

Read about iou_thresh=0.213:

Our loss initially only paired the best anchor (out of all 9) to each target. We changed this to pair a target to any anchor above iou_thres = 0.213 (all paired anchors generate losses). So for example all 9 anchors might be paired or none at all. This change boosted mAP by about 0.07 to 0.47.

https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-553159098

[LukeAI]

Or you can add different values for different yolo-layers: scale_x_y=1.05 (for 17x17) , 1.1 (34x34) 1.2 (68x68).

Not sure what you mean by 17x17 etc. yolo layers - is that the scale on which they detect? Is 17x17 the first one in the cfg?

Read about iou_thresh=0.213: So should I set iou_thresh=0.213 or 3? which one is more promising IYO?

[AlexeyAB]

Not sure what you mean by 17x17

Yolo layer size.

Use scale_x_y=1.05 , 1.1, 1.2 in different layers as usual.

So should I set iou_thresh=0.213 or 3? which one is more promising IYO?

I don't know.

[LukeAI]

ok - should I be using giou loss for the iou threshold to be relevant - or is it independent of the loss function?

[AlexeyAB]

independent

[LukeAI]

OK yolo_gauss_spp_swish is looking strong. I've ended the training early to add iou and scale and start again. will report back shortly...

[LukeAI]

Here is yolo_gaussian_spp_swish_scale_iou.cfg I went for iou_threshold=0.25

The addition of the IOU seemed to help it converge faster but not really clear if it ultimately led to higher mAP. Need to finish the above experiment.

I notice that the mAP doesn't stabilise until the learning rate steps down - around 40000 and 45000 - I'm not really sure how to interpret this - do you think that it might be better to step down the learning rate much earlier - maybe at 10,000 and 15000 or something?

yolo_gaussian_spp_swish_iou_scale.cfg.txt

[AlexeyAB]

I notice that the mAP doesn't stabilise until the learning rate steps down - around 40000 and 45000 - I'm not really sure how to interpret this - do you think that it might be better to step down the learning rate much earlier - maybe at 10,000 and 15000 or something?

How many images do you have in training dataset? If you have only 20 000 images then set max_batches=20000 and steps at 15000, 18000

The addition of the IOU seemed to help it converge faster but not really clear if it ultimately led to higher mAP. Need to finish the above experiment.

Yes, may be it is only for the fast training, but not for the high mAP. Try Gaussian_yolo + swish, without iou_thresh. while the mAP increases.

Also try Gaussian_yolo + CIOU + swish

iou_normalizer=0.25
cls_normalizer=1.0
iou_loss=ciou

[AlexeyAB]

It is interesting, is Gaussian_yolo + CIOU better than Gaussian_yolo

[LukeAI]

My training set is only 14000 - I've edited the .cfg for steps at 18000, 19000 max_batched=22000 - going to see how that works out. I spliced the two graphs of the training without iou_thresh together above. So yeah, in this case iou_thresh=0.25 led to faster convergence but not higher final mAP - but my steps and max batches config wasn't that sensibly configured. I'll try CIOU shortly and report back. [EDIT] OK yeah I can confirm that using sensible batches and steps params gave me convergence to a higher mAP of 74%

[LukeAI]

Setting CIOU hurt mAP substantially in my case:

iou_loss=ciou

Without CIOU

But I didn't set iou_normalizer=0.25 cls_normalizer=1.0 Is that needed for CIOU?

[AlexeyAB]

@LukeAI

Did you test [Gaussian_yolo] + CIoU or [yolo]+CIoU ?

Yes, they use https://github.com/Zzh-tju/DIoU-darknet/blob/master/cfg/coco-ciou.cfg#L811-L812

iou_normalizer=0.5
cls_normalizer=1.0

---

[EDIT] OK yeah I can confirm that using sensible batches and steps params gave me convergence to a higher mAP of 74%

What do you mean?

[glenn-jocher]

@LukeAI normalizers should not have much effect. As long as you use the same values between both comparisons then it should be a valid (apples to apples) comparison.

In my study I found CIoU and DIoU produced about the same results as GIoU. Actually I think I saw that CIoU/DIoU may produce slightly better results in the very early epochs (or on smaller datasets for more epochs), but on larger datasets with lots of iterations (COCO) it did not help at all.

[LukeAI]

This is the result for CIOU with iou_normalizer=0.25 cls_normalizer=1.0 A loss of about 1mAP compared to without CIOU or normalizers.

[AlexeyAB]

@LukeAI @glenn-jocher

It seems that it necessary to reduce iou_normalizer= from 1.0 to 0.5 - 0.1 if there is used GIoU, DIoU or CIoU.

[LukeAI]

OK using iou_normalizer=0.25 cls_normalizer=1.0 with gaussian_spp_swish (without iou_thresh) and with the default loss function has given me the best result so far - the normalisers added +1mAP

yolo_gaussian_spp_swish_normalisers.cfg.txt

[AlexeyAB]

@LukeAI

iou_normalizer= will affect only for losses C/D/GIoU: https://github.com/AlexeyAB/darknet/blob/31d483a2950a4d747b31f033e5725bb8de50ff1e/src/gaussian_yolo_layer.c#L243-L286

[LukeAI]

oh! so my improved mAP was just from cls_normalizer=1.0 ? Or... just random?

[AlexeyAB]

@LukeAI Just random.

cls_normalizer=1.0 is by default.

[anamozov]

I found original yolov3 and Gaussian yolo gave me almost the same mAP after more after around 35000 iterations for 54 class problem. I could't wait to finish training due to time problems. Gaussian_yolo yolov3

[nyj-ocean]

@AlexeyAB I use my own dataset to train several models in this ropo. I train 70k steps. My dataset:1500 images for trainning set,other 1500 images for val set. I use val set to calculate mAP.

	best mAP	Recall
yolov3	85.63	77
yolov3+Gaussian	85.97	80
yolov3+CIoU	85.29	75
yolov3+Gaussian+CIoU	86.4	81

yolov3+Gaussian+CIoU gets a 0.77% improvement in mAP and a 4% improvement in Recall compared to yolov3. I want to know these improvement in mAP and Recall of yolov3+Gaussian+CIoU are just for random or for the added module Gaussian+CIoU?

[AlexeyAB]

@nyj-ocean

• Did you get Recall by using ./darknet detector map. ... command? You should compare Recall only if the same Precision (so you should play with -thresh 0.x flag).

I want to know these improvement in mAP and Recall of yolov3+Gaussian+CIoU are just for random or for the added module Gaussian+CIoU?

Just train again yolov3 and yolov3+Gaussian+CIoU

Also show chart.png of both models.

[AlexeyAB]

@LukeAI

iou_normalizer= will affect only for losses C/D/GIoU:

I fixed it.

Now iou_normalizer=0.1 and uc_normalizer=0.1 in [Gaussian_yolo] affect even without C/D/GIoU.

[AlexeyAB]

@WongKinYiu

It seems that we should use

1. iou_loss=1 for [yolo]
2. iou_loss=0.01 for [yolo] + C/D/GIoU (may be 0.07 )
3. iou_loss=0.1 and uc_loss=0.1 for [Gaussian_yolo] (may be 0.3 )
4. iou_loss=0.01 and uc_loss=0.01 for [Gaussian_yolo] + C/D/GIoU (may be 0.07 )

I won’t be surprised if the main effect of improving the AP75 (and decreasing AP50) is not from the good GIoU algorithm, but just from higher values ​​of iou_loss. I.e. the same effect we can achieve by using default [yolo] with default mse-loss but with iou_normalizer=10 or 100

---

1. [yolo] has class_loss and iou_loss ~= 10

---

2. [yolo]+GIoU has class_loss and iou_loss ~= 1000

---

2. [Gaussian_yolo] has high iou_loss and uc_loss ~100

---

3. [Gaussian_yolo] + GIoU has much higher iou_loss and uc_loss ~1000

---

[HagegeR]

so just multiplying the error is what make the network learn faster, but I think the way the loss is calculated result in better ap75

[AlexeyAB]

@HagegeR
A higher iou_loss shifts the priorities to more accurate IoU (coordinates and sizes) of the object (increases AP75), but to a less accurate prediction of the class_id (decreases AP50).

C/D/GIoU increases AP75, but decreases AP50.

[glenn-jocher]

Yes, normalizers have a tremendous effect on performance. In our repo we've spent thousands of GPU hours evolving these normalizers on COCO along with all of our other hyperparameters. See https://github.com/ultralytics/yolov3/issues/392

Our 3 main balancing hyperparameters are here, though they are changing every few weeks as new evolution results come in. They must change depending on image size, as well as class count, and occupancy (how many objects per image).

hyp = {'giou': 3.31,  # giou loss gain
       'cls': 42.4,  # cls loss gain
       'obj': 52.0,  # obj loss gain (*=img_size/416 if img_size != 416)}

[glenn-jocher]

I have updated mAPs now from https://github.com/ultralytics/yolov3#map using the default hyperparameters and default training settings on COCO2014, starting yolov3-spp.cfg from scratch.

	Size	COCO mAP @0.5...0.95	COCO mAP @0.5
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	320	14.0 28.7 30.5 35.4	29.1 51.8 52.3 54.3
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	416	16.0 31.2 33.9 39.0	33.0 55.4 56.9 59.2
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	512	16.6 32.7 35.6 40.3	34.9 57.7 59.5 60.6
YOLOv3-tiny YOLOv3 YOLOv3-SPP YOLOv3-SPP ultralytics	608	16.6 33.1 37.0 40.9	35.4 58.2 60.7 60.9

[AlexeyAB]

@glenn-jocher

is YOLOv3-SPP with default hyperparameters giou=1 cls=1 obj=1, while YOLOv3-SPP ultralytics with these hyperparameters?

hyp = {'giou': 3.31, # giou loss gain 'cls': 42.4, # cls loss gain 'obj': 52.0, # obj loss gain (*=img_size/416 if img_size != 416)}

So good hyper parameters for GIoU:

[yolo]
iou_normalizer=0.07    # giou hyperparameter: 3.31 / ((52+42)/2) = 0.07
cls_normalizer=1.0     # obj hyperparameter
iou_loss=giou

---

img_size/416 is it image size or network size? Since every image will be resized to the network size.

[glenn-jocher]

@AlexeyAB 416 is the network size. So for example if the network is trained at 320, the objectness hyperparameter would be multiplied by 320/416. I found this helped keep the balance when network size changed.

Yes, roughly speaking we found an optimal GIoU gain is about 10 times smaller than cls and obj, and that obj and cls seem to optimally be of similar magnitude. I average the elements in each yolo layer, and sum the 3 values for the 3 yolo layers. So for example obj loss = mean(obj_loss_yolo_layer1) + mean(obj_loss_yolo_layer2) + mean(obj_loss_yolo_layer3). The chart below shows the loss after multiplying by the hyps:

[glenn-jocher]

I think the fundamental concept is that if each of the 3 loss components are equally important to the solution (and I believe they are), then they should roughly each be represented equally in the total loss (i.e. 1/3, 1/3, 1/3). This is probably the best place to start when in doubt, and then fine tune from there if you can.

[AlexeyAB]

@glenn-jocher

hyp = {'giou': 3.31, # giou loss gain 'cls': 42.4, # cls loss gain 'obj': 52.0, # obj loss gain (*=img_size/416 if img_size != 416)}

Are these hyperparameters good for C/D/GIoU only, or also the same values for the default MSE-loss?