search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.65k stars 7.96k forks source link

Inquiries about yolov3 and yolov3-tiny model accuracy #8078

Open cig2982 opened 3 years ago

cig2982 commented 3 years ago

Hi I discovered a strange thing while measuring mAP on the coco validation set using the yolov3 and yolov3-tiny models.

For yolov3, 66.19% is measured for the validation set as shown in the picture below. yolov3 This is a higher result than the result measured at 55.3% for the test set specified on the https://pjreddie.com/darknet/yolo/ homepage. (And is 66.19% too high compared to 55.3%? )

However, for yolov3-tiny, 19.41% is measured as shown in the figure below, which is very small compared to 33.1% measured for the test set. yolov3-tiny In my opinion, yolov3-tiny should also have a higher mAP measured on the validation set than on the test set. Please tell me why this is happening. Maybe you uploaded the wrong yolov3-tiny model?

We would appreciate it if you could respond to the above.

lsd1994 commented 3 years ago

What command line do you use?

cig2982 commented 3 years ago

What command line do you use? I used the below command

./darknet detector map *.data *.cfg *.weights

The data used is converted from val2017.json of coco dataset into class_label bbox(x,y,w,h) format used in darknet.

louielu1027 commented 2 years ago

@cig2982 Do you know the reason ? ？ I got the same results.......when I test on testdev2017, yolov3-tiny only got map@0.5 0.183.
scores.txt : AP: 0.088 AP_50: 0.183 AP_75: 0.075 AP_small: 0.001 AP_medium: 0.055 AP_large: 0.227

louielu1027 commented 2 years ago
  1. change mask = 0,1,2 to mask = 1,2,3.
  2. add letter_box=1 in [net]

but i dont know why.........

mwessley commented 2 years ago

  1. change mask = 0,1,2 to mask = 1,2,3.

    1. add letter_box=1 in [net]

but i dont know why......

it seems that back then the original TinyYolov3 weights you can download were trained on mask = 1,2,3 than later on the "mistake" was corrected in the mask but the weights were never updated.