search

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )
http://pjreddie.com/darknet/
Other
21.75k stars 7.96k forks source link

How do we print mAP, FN/FP and loss? #4339

Closed daddydrac closed 4 years ago

daddydrac commented 4 years ago

How do we print mAP, FN/FP and loss?

AlexeyAB commented 4 years ago

https://github.com/AlexeyAB/darknet#when-should-i-stop-training

darknet.exe detector map data/obj.data yolo-obj.cfg backup\yolo-obj_9000.weights

darknet.exe detector train data/obj.data yolo-obj.cfg darknet53.conv.74 -map

daddydrac commented 4 years ago

@AlexeyAB - Ok my command needs to look something like this, bc I am using yolo-spp w/ coco:

./darknet detector test cfg/coco.data cfg/yolov3-spp.cfg weights/yolov3-spp.weights

Also, how do I output a batch of images w/ bounding boxes along with mAP?

daddydrac commented 4 years ago

@AlexeyAB - plz see above, i am not sure how to make it work with yolov3-spp.

When I run what you posted here: https://github.com/AlexeyAB/darknet/issues/2746 -> I get this error:

calculation mAP (mean average precision)...
Couldn't open file: coco_testdev
AlexeyAB commented 4 years ago

You should get MS COCO dataset by using: https://github.com/AlexeyAB/darknet#datasets Then you should specify correct path in coco.data file to the train and valid datasets in my case:

classes= 80
train  = E:/MSCOCO/trainvalno5k.txt
valid  = E:/MSCOCO/5k.txt
names = data/coco.names
backup = backup
eval=coco
daddydrac commented 4 years ago

Thank you, I will try this.

daddydrac commented 4 years ago

Thank you for that, it worked fine. My next question is how do we connect our own images/data to it?

This all I get back as well , everything is 0:

Can't open label file. (This can be normal only if you use MSCOCO): /labels/val2014/COCO_val2014_000000581899.txt 

 detections_count = 40000, unique_truth_count = 0  
class_id = 0, name = person, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 1, name = bicycle, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 2, name = car, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 3, name = motorbike, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 4, name = aeroplane, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 5, name = bus, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 6, name = train, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 7, name = truck, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 8, name = boat, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 9, name = traffic light, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 10, name = fire hydrant, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 11, name = stop sign, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 12, name = parking meter, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 13, name = bench, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 14, name = bird, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 15, name = cat, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 16, name = dog, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 17, name = horse, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 18, name = sheep, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 19, name = cow, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 20, name = elephant, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 21, name = bear, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 22, name = zebra, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 23, name = giraffe, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 24, name = backpack, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 25, name = umbrella, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 26, name = handbag, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 27, name = tie, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 28, name = suitcase, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 29, name = frisbee, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 30, name = skis, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 31, name = snowboard, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 32, name = sports ball, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 33, name = kite, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 34, name = baseball bat, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 35, name = baseball glove, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 36, name = skateboard, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 37, name = surfboard, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 38, name = tennis racket, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 39, name = bottle, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 40, name = wine glass, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 41, name = cup, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 42, name = fork, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 43, name = knife, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 44, name = spoon, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 45, name = bowl, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 46, name = banana, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 47, name = apple, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 48, name = sandwich, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 49, name = orange, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 50, name = broccoli, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 51, name = carrot, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 52, name = hot dog, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 53, name = pizza, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 54, name = donut, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 55, name = cake, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 56, name = chair, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 57, name = sofa, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 58, name = pottedplant, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 59, name = bed, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 60, name = diningtable, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 61, name = toilet, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 62, name = tvmonitor, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 63, name = laptop, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 64, name = mouse, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 65, name = remote, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 66, name = keyboard, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 67, name = cell phone, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 68, name = microwave, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 69, name = oven, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 70, name = toaster, ap = 0.00%        (TP = 0, FP = 0) 
class_id = 71, name = sink, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 72, name = refrigerator, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 73, name = book, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 74, name = clock, ap = 0.00%      (TP = 0, FP = 0) 
class_id = 75, name = vase, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 76, name = scissors, ap = 0.00%       (TP = 0, FP = 0) 
class_id = 77, name = teddy bear, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 78, name = hair drier, ap = 0.00%     (TP = 0, FP = 0) 
class_id = 79, name = toothbrush, ap = 0.00%     (TP = 0, FP = 0) 

 for conf_thresh = 0.25, precision = -nan, recall = -nan, F1-score = -nan 
 for conf_thresh = 0.25, TP = 0, FP = 0, FN = 0, average IoU = 0.00 % 

 IoU threshold = 50 %, used 101 Recall-points 
 mean average precision (mAP@0.50) = 0.000000, or 0.00 % 
Total Detection Time: 83.000000 Seconds
daddydrac commented 4 years ago

Please close this out. I have gone to a diff Yolo/COCO solution I found on GitHub that has better documentation, and have made enough progress that I’m almost finished.

AlexeyAB commented 4 years ago

i am on linux, obviously

Run on your custom dataset:

./darknet detector train data/obj.data yolo-obj.cfg darknet53.conv.74 -map

Can't open label file. (This can be normal only if you use MSCOCO): /labels/val2014/COCO_val2014_000000581899.txt

Find files like COCO_val2014_000000581899.txt and put them to the directory: /labels/val2014/