search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.78k stars 7.96k forks source link

Here you can post your trained models on different Datasets - 3 files: cfg, weights, names #3874

Open AlexeyAB opened 5 years ago

AlexeyAB commented 5 years ago

Here you can post your trained models on different Datasets - 3 files:

  1. cfg
  2. weights
  3. names
  4. Optional: accuracy mAP@0.5 or/and mAP@0.5...0.95
  5. Optional: BFLOPS and/or Inference time (milliseconds)

COCO test-dev

Model Size BFLOPS Inference time, ms AP@.5:.95 AP@.5 AP@.75 URL
yolo_v3_tiny_pan3 aa_ae_mixup scale_giou (no sgdr).txt 416x416 8.4 6.4 18.8% 36.8% 17.5% https://github.com/AlexeyAB/darknet/issues/3708#issuecomment-532070211
yolov3-tiny-prn.cfg.txt 416x416 3.5 3.8 - 33.1% - URL
enet-coco.cfg.txt 416x416 3.7 22.7 - 45.5% - URL

ImageNet valid

Model BFLOPs Inference Time (ms) Top1, % URL
shufflenetv2 and weights 0.375 32 52% URL
efficient_b0 and weights 0.915 76 71% URL
MobileNet v2 and weights 0.85 - 67% URL
gameliee commented 5 years ago

I would try Dataset: Visdrone2019 1.cfg yolov3-tiny_3l.cfg.txt

  1. weights https://drive.google.com/file/d/1fqWdpYpTJkKLnVhEiQQqm_SNvMp_qSE0/view?usp=sharing
  2. names visdrone2019.names.txt

Hope to see comments on these

dexception commented 5 years ago

MobileNet V2 Imagenet Top 1: 0.670320, top 5: 0.876620 Iterations: 600,000

https://github.com/AlexeyAB/darknet/files/3310574/mobilenet_v2.cfg.txt imagenet1k.data

mobilenetv2_last.weights https://drive.google.com/file/d/1lW87XQtZIYKIqu8DHvJwBUupJBh4Zpdl/view?usp=sharing

AlexeyAB commented 5 years ago

@ntd94 Thanks. What mAP (accuracy) did you get on Visdrone2019 val/test?

hasirk commented 5 years ago

  1. cfg: yolo-obj.cfg.txt

  2. weights: https://bit.ly/2oa9Rt3

  3. names

    • bike
    • bus
    • car
    • motor
    • person
    • rider
    • traffic light
    • traffic sign
    • train
    • truck

  4. results: https://bit.ly/2nxWL8h

ChenCong7375 commented 5 years ago

Any yolo_v3_tiny_pan3 aa_ae_mixup scale_giou weights? @AlexeyAB

laclouis5 commented 5 years ago

Dataset

Custom small dataset with 6 classes (crop and stems), images are approx. 2 MPix with a 4/3 ratio.

For each class are tagged the whole plant and its stem. The stem annotation is a square box approx. the same relative size for each class and image.

There is large overlap for some bounding boxes, some plants are very small and others almost fit half the image.

database

Class Images Annotations
Bean 221 1118
Bean Stem 221 1176
Maize 489 889
Maize Stem 489 927
Leek 197 855
Leek Stem 197 927

Results

Model is trained for 10 000 steps using original parameters excepted for subdivisions that is adjusted to fit on the GPU (Nvidia GTX 1060 6GB). For old models (Tiny, Tiny 3L, ...) width and height are changed to 544 instead of 416.

I Will update this post as soon as I have new results. I will also try to compare AlexeyAB Darknet implementation with other realtime frameworks (CenterNet, Ultralytics, RetinaNet, ...).

Model Config Files Training Chart mAP@0.5 (scratch / pre-trained)* mAP@[0.5...0.95] (scratch / pre-trained)* FPS
Yolo v4 (random=0, resize=1.5) cfg chart_yolov4-custom — / 90.00% — / 50.73% 19fps
Yolo V3 CSR Spp Panet Optimal cfg chart — / 88.30% — / 48.42% 18fps
Yolo v4 Tiny cfg chart — / 88.76% — / 46.74% 146fps
Yolo V3 CSR Spp Panet cfg chart 83.38% / 88.34% 42.21% / 45.88% 18fps
Tiny Yolo V3 Pan 3 cfg chart 87.16% / 87.46% 42.18% / 42.41% 65fps
Tiny Yolo V3 Pan Mixup cfg chart 2 88.06% 40.00% 68fps
Tiny Yolo V3 Gaussian Matrix cfg chart 2 86.84% 40.09% —
Tiny Yolo V3 3L cfg chart 83.93% / 84.41% 38.12% / 38.15% 120fps
Tiny Yolo V3 Prn cfg chart 3 82.93% 33.82% 180fps
Tiny Yolo V3 cfg chart 82.39% 33.28% 140fps
Yolo V3 Spp Pan Scale cfg chart 75.95% 31.76% 13fps

* Using pre-trained weights for deep networks such as yolo v3 and CSR significantly improves training stability and speed as well as the accuracy. This effect is less significant for shallower networks such as Tiny Yolo v3 Pan3.

Model Config Files Training Chart mAP@0.5 mAP@[0.5...0.95] FPS
CenterNet dla-34 512x512* — — 75.7% 41.6% 25fps
Ultralytics Yolov3-SPP 512x512 — — — —

* _Using pre-trained weights from ctdet_coco_dla_2x.pth_

Kyuuki93 commented 4 years ago

DataSet was one-class custom data with 33216 images for training and 4324 images for validation.

Experimental results will be updated continuously.

Model CFG AP@.5 AP@.75
spp,mse cfg 89.52% 51.72%
spp,mse,it=0.213 - 92.01% 60.49%
spp,giou(in=0.07) - 89.55% 51.25%
spp,giou(in=0.5) - 90.08% 59.55%
spp,ciou(in=0.5) - 89.49% 58.39%
spp,giou,gs(in,un=0.5) cfg 91.39% 58.01%
spp,ciou,gs(in,un=0.5) - 89.81% 59.01%
spp,giou,gs,mixup(in,un=0.5) - 89.51% 58.73%
spp,giou,gs,mosaic(in,un=.5) - 90.48% 60.02%
spp,giou,gs(in,un=0.5,it=0.213) - 91.89% 63.53%
spp,giou,gs,swish(in,un=0.5,it=0.213) - 91.54% 60.70%
spp,giou,gs,mosaic(in,un=0.5,it=0.213) - 91.82% 63.89%
spp,mse,it=0.213,asff(in,un=0.5,it=0.213) cfg 92.45% 61.83%
spp,giou,it=0.213,asff(in,un=0.5,it=0.213) - 92.33% 63.74%
spp,giou,it=0.213,asff(softmax),rfb(bn=0) - 91.57% 64.95%
spp,giou,it=0.213,asff(softmax),rfb(bn=1) - 92.32% 60.05%
spp,giou,it=0.213,asff(softmax),dropblock(size=7) ,rfb(bn=0) - 91.65% 61.35%
spp,giou,it=0.213,asff(softmax),dropblock(size=7) ,rfb(bn=1) - 92.12% 63.88%

A summary,

method AP@.5 AP@.75
iou_thresh + +
mse -> giou - +
giou -> ciou - +
gaussian_yolo + -
mixup - -
moasic = +
swish - -
asff + +
rfb ? ?
dropblock ? ?
sctrueew commented 4 years ago

@Kyuuki93 Hi, Can you share the spp,giou,gs,mosaic(in,un=0.5,it=0.213) cfg file?

Thanks!

Kyuuki93 commented 4 years ago

@Kyuuki93 Hi, Can you share the spp,giou,gs,mosaic(in,un=0.5,it=0.213) cfg file?

Thanks!

just download spp,giou,gs(in,un=0.07), then

  1. change iou_normalizer and uc_normalizer to 0.5 which now is 0.07
  2. add mosaic = 1 in [net] session
  3. add iou_thresh = 0.213 in all [Gaussion_yolo] session
Kyuuki93 commented 4 years ago

In my application, precision, recall, AP@0.5 with high thresh, e.g. thresh = 0.7 or 0.85, were more important, based on https://github.com/AlexeyAB/darknet/issues/3874#issuecomment-561064425, create a new table here,

Model AP@.5 AP@.75 precision (th=0.85) recall(th=0.85) precision (th=0.7) recall(th=0.7)
spp,mse 89.50% 51.75% 0.98 0.20 0.97 0.36
spp,giou 90.09% 59.55% 0.98 0.25 0.97 0.40
spp,ciou 89.88% 58.39% 0.99 0.22 0.97 0.38
spp,giou,gs 91.39% 58.01% 0.99 0.05 0.97 0.47
spp,ciou,gs 89.82% 59.01% 0.99 0.02 0.99 0.24
spp,giou,gs,mixup 89.51% 58.73% 0.98 0.01 0.98 0.20
spp,giou,gs,mosaic 90.47% 63.89% 1.00 0.01 0.99 0.22
spp,mse,it 92.01% 60.49% 0.97 0.60 0.95 0.72
spp,giou,it 91.79% 63.09% 0.96 0.57 0.95 0.71
spp,giou,it,igt=0.85 92.01% 62.84% 0.98 0.37 0.97 0.55
spp,giou,it,igt=1.0 91.43% 61.82% 0.98 0.21 0.97 0.43
spp,giou,it,tt=0.7 nan - - - - -
spp,giou,it,tt=0.85 78.01% 28.27% 0.97 0.06 0.95 0.21
spp,giou,gs,it 91.87% 63.53% 0.99 0.16 0.97 0.52
spp,mse,it,asff 92.49% 61.83% 0.97 0.57 0.95 0.70
spp,giou,it,asff
spp,giou,it,asff,rfb 92.32% 60.05% 0.98 0.43 0.96 0.62
csresnext50-panet,spp,giou,it 92.80% 64.16% 0.97 0.51 0.96 0.67
ramdhan1989 commented 4 years ago

I would try Dataset: Visdrone2019 1.cfg yolov3-tiny_3l.cfg.txt

  1. weights https://drive.google.com/file/d/1fqWdpYpTJkKLnVhEiQQqm_SNvMp_qSE0/view?usp=sharing
  2. names visdrone2019.names.txt

Hope to see comments on these

I tried to my own drone video. the result is really good. Please advice how to get the similar performance ? for my purposes, I need to separate truck category into several truck type, person, car, bus and motorcycle. is there any dataset that could be used especially for aerial view image ?

titanbender commented 4 years ago

@laclouis5: What is the difference between

mAP@0.5 (scratch / pre-trained)* and mAP@[0.5...0.95] (scratch / pre-trained)*?

From how I read it the mAP is both calculated at 50%.

laclouis5 commented 4 years ago

@titanbender,

No, mAP@[0.5...0.95] is the average AP over ten different IoU thresholds ranging from 0.5 to 0.95 by 0.05 increment.

DayBreak-u commented 4 years ago

Here you can post your trained models on different Datasets - 3 files:

  1. cfg
  2. weights
  3. names
  4. Optional: accuracy mAP@0.5 or/and mAP@0.5...0.95
  5. Optional: BFLOPS and/or Inference time (milliseconds)

COCO test-dev

Model Size BFLOPS Inference time, ms AP@.5:.95 AP@.5 AP@.75 URL yolo_v3_tiny_pan3 aa_ae_mixup scale_giou (no sgdr).txt 416x416 8.4 6.4 18.8% 36.8% 17.5% #3708 (comment) yolov3-tiny-prn.cfg.txt 416x416 3.5 3.8 - 33.1% - URL enet-coco.cfg.txt 416x416 3.7 22.7 - 45.5% - URL ImageNet valid

Model BFLOPs Inference Time (ms) Top1, % URL shufflenetv2 and weights 0.375 32 52% URL efficient_b0 and weights 0.915 76 71% URL MobileNet v2 and weights 0.85 - 67% URL

mobilenetv2 cfg file is error, stride 16 need repeat 7 times,but it‘s stride 8 repeat 7 times, this add the bflops

colinlin1982 commented 4 years ago

Pascal VOC 2007+2012 yolov3-tiny-prn with mosaic and ciou, input_size 352x352, 2.45BFlops, map@0.5=64.26% (backbone from yolov3-tiny-prn.weights for coco). yolov3-tiny-prn.cfg.txt and weights yolov3-tiny-prn_352@64.26.weights.zip

usamahjundia commented 4 years ago

@laclouis5 hello there, is there a list / compilation of all those custom configs you used? i cannot seem to find them on cfg/ folder

laclouis5 commented 4 years ago

@laclouis5 hello there, is there a list / compilation of all those custom configs you used? i cannot seem to find them on cfg/ folder

Hi @usamahjundia, all cfg files are attached in https://github.com/AlexeyAB/darknet/issues/3874#issuecomment-549470673, 2nd column.

usamahjundia commented 4 years ago

@laclouis5 hello there, is there a list / compilation of all those custom configs you used? i cannot seem to find them on cfg/ folder

Hi @usamahjundia, all cfg files are attached in #3874 (comment), 2nd column.

Thanks, but that is unfortunately not what i meant

What i meant was, where did you discover them in the first place? or are they custom-made?

Sorry for the confusion

laclouis5 commented 4 years ago

@usamahjundia They are regular cfg files developed in this repo, I did not customise them. You can found them in the cfg folder of the repo.

ShaneHsieh commented 4 years ago

@laclouis5 hello what is different with (scratch / pre-trained)? and Tiny Yolo V3 Pan 3 is show 87.16% / 87.46%, but the Training Chart is show the best map is 89%,this is why? thank.

laclouis5 commented 4 years ago

@ShaneHsieh Error on my side, I updated the post with the good image.

Pre-trained is for models trained with initial weights from models trained on MS Coco. See the training tutorial of this repo for more information on how to do that. From scratch is without pre-trained weights.

ShaneHsieh commented 4 years ago

@laclouis5 Thank.

dog-qiuqiu commented 4 years ago

MobileNetV2-YOLOv3-Lite&Nano Darknet

Mobile inference frameworks benchmark (4*ARM_CPU)

Network VOC mAP(0.5) COCO mAP(0.5) Resolution Inference time (NCNN/Kirin 990) Inference time (MNN arm82/Kirin 990) FLOPS Weight size
MobileNetV2-YOLOv3-Lite 72.61 36.57 320 33 ms 18 ms 1.8BFlops 8.0MB
MobileNetV2-YOLOv3-Nano 65.27 30.13 320 13 ms 5 ms 0.5BFlops 3.0MB
MobileNetV2-YOLOv3-Fastest 46.55 & 320 8.2 ms 2.4 ms 0.13BFlops 700kb

https://github.com/dog-qiuqiu/MobileNetv2-YOLOV3