Closed AlexeyAB closed 3 years ago
Implemented.
Something goes wrong with
[net]
mosaic=1
chart: cfg file: yolov3-spp-giou-gs-mos.cfg.txt
without mosaic chart:
For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.
@LukeAI @Kyuuki93
I improved mosaic=1 for Detector. Try new mosaic. Two commits: https://github.com/AlexeyAB/darknet/commit/87f36b79f54ac0b84512a3c1c4284d31d100f6f7 and https://github.com/AlexeyAB/darknet/commit/13f064f1be1ccf28fae002349ba53fbf61a122a5
@LukeAI @Kyuuki93
I improved mosaic=1 for Detector. Try new mosaic. Two commits: 87f36b7 and 13f064f
results were updated here https://github.com/AlexeyAB/darknet/issues/3874#issuecomment-561064425
@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75
58.01% AP@75
60.02% AP@75
Also it seems that iou_n=0.07
is too low value for GIoU, and iou_n=0.5
is much better.
@Kyuuki93 It seems that new mosaic=1 significantly improves AP@75
- spp,giou,gs(iou_n=0.50,uc_n=0.50) | - | | 91.39% AP@50 |
58.01% AP@75
- spp,giou,gs,mosaic(n=0.50,uc_n=0.50) | - | | 90.48% AP@50 |
60.02% AP@75
Also it seems that
iou_n=0.07
is too low value for GIoU, andiou_n=0.5
is much better.
And I updated mosaic
results iou_thresh = 0.213
, the improvement at AP@75 still exists but not that significant
91.89%
AP@50 | 63.53%
AP@75|91.82%
AP@50 | 63.89%
AP@75|@Kyuuki93 So both iou_thresh=0.213 and mosaic=1 increase accuracy. Maybe mosaics will give a greater increase in accuracy on large datasets
For me, mosaic gave me -0.4% mAP - don't really know if that's just random fluctuation or it did actually hurt performance a little.
Speaking of training accuracy fluctuation I did an experiment on a small private dataset (see #3874 (comment) for dataset details).
I trained 15 instances of "Yolo v3 Tiny 3L" for 10_000 steps on the same dataset with same config file and parameters.
The randomness of training brings some fluctuation in accuracy (mAP@.5):
Std Dev and extend are higher for classes harder to detect.
@laclouis5
What pre-trained weights-file did you use for training yolo_v3_csresnext50-pan-spp.cfg
there? https://github.com/AlexeyAB/darknet/issues/3874#issuecomment-549470673
@AlexeyAB
I don't use any pre-trained weights in all my experiments, every network is trained from scratch.
@laclouis5 (I don't use any pre-trained weights) - This can reduce accuracy, the larger the model, the more pre-trained weights are needed.
@AlexeyAB
I try to train my networks in a very similar context to obtain a fair comparison. I don't fully control and understand how pre-trained weights affect initialisation and training as pre-trained weights could be different between networks (Tiny and plain Yolo for instance), so I prefer to train from scratch to compare apples to apples.
I also observe that on small database and small network (Yolo v3 Tiny) training curve is steeper but the final accuracy (mAP@.5) is roughly the same as it plateaus rapidly near 3_000 iterations in my case.
In the following image are overlapped training curves with and without pre-trained weights (from a network I previously trained).
Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg
and/or with bigger databases with more classes.
That being said, when deploying a network I would certainly train with Coco weights to get the best result.
@AlexeyAB I'll train yolo_v3_csresnext50-pan-spp.cfg
with pre-trained weights to quantify the difference on this one.
@laclouis5
Maybe pre-trained weights / transfert learning is more relevant and can increase accuracy for deeper networks such as yolo_v3_csresnext50-pan-spp.cfg and/or with bigger databases with more classes.
I think so.
I'll train yolo_v3_csresnext50-pan-spp.cfg with pre-trained weights to quantify the difference on this one.
Yes, try and compare accuracy.
Is ok to use mosaic=1 and mixup=1 at the same time? @AlexeyAB
@tianfengyijiu Yes, but will be used only Mosaic.
@AlexeyAB Here is the result:
Model | Training Chart | mAP@.5 | mAP@[0.5...0.95] |
---|---|---|---|
Yolo v3 Spp Panet | 83.38% | 42.21% | |
Yolo v3 Spp Panet with pre-trained weights | 88.34% | 45.88% |
The pre-trained version is much more stable, +0.30% mAP@0.5 and +3% mAP@[0.5...0.95] higher than my best non pre-trained network.
If I have time l'll train other networks with pre-trained weights to be able to compare this result against.
@AlexeyAB ,
Continuing issue with mosaic
in Detector experienced here : https://github.com/AlexeyAB/darknet/issues/4432#issuecomment-570817372
I'm indeed using the newest repo and the issue seems to only occur when mosaic
is used with letter_box
and images are positioned in such a way that they overlap awkwardly. All other images come out correctly.
For example :
This issue goes away when turning off letter_box
as this prevents images running over one another, it seems. For example :
@BernoGreyling Thanks! Yes, it seems there is an issue for combination letter_box=1 mosaic=1
hi @AlexeyAB
Is this problem solved ? ( combination letter_box=1 mosaic=1
)
Not yet
can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX
can you share the paper for mosaic, i want cite, but not find in Google Scholar, THX
mosaic cames from yolov3 or yolov4
@BernoGreyling Thanks! Yes, it seems there is an issue for combination
letter_box=1 mosaic=1
Hi @AlexeyAB Has this bug been fixed yet?
@calves91
Yes,
Just use
[net]
mosaic=1
mosaic_bound=1
letter_box=1
In my experiments, usually this "bug" improves accuracy, so usually it is better to train without mosaic_bound=1
, because in the most cases we need to detect the whole object:
mosaic_bound=1
- increases accuracy for occluded objects (it detects only visible part of object)mosaic_bound=0
- increases accuracy for not-occluded objects (it detects visible+occluded parts of object) - usually it is more important, because we have much more not-occluded objects than occludedhttps://github.com/AlexeyAB/darknet/wiki/CFG-Parameters-in-the-%5Bnet%5D-section
mosaic_bound=1 - limits the size of objects when mosaic=1 is used (does not allow bounding boxes to leave the borders of their images when Mosaic-data-augmentation is used)
Mosaic-data augmentation (like CutMix augmentation): https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-552158041
Mosaic-data mAP increasing: https://github.com/AlexeyAB/darknet/issues/3114#issuecomment-552158041
comparison MixUp and CutMix: https://github.com/AlexeyAB/darknet/issues/3272#issuecomment-541031314
Use:
Currently works only if Darknet is compiled with OpenCV.
New mosaic:
Old mosaic=1 - CLICK ME to view
![image](https://user-images.githubusercontent.com/4096485/70151300-7329bf80-16bc-11ea-8eb7-1829c74567dc.png)