YOLOv3 low accuracy on person & fire detection

[ahmadalzoubi13579]

i already read this issue #5257

i trained Yolov3 to detect person and fire objects in images , my dataset division :

Train :

• total images : 4773

• number of images per class :

  • person : 774
  • fire : 1869

• number of objects per class :

  • person : 1739
  • fire : 2926

Validation :

• total images : 1140

• number of images per class :

  • person : 184
  • fire : 413

• number of objects per class :

  • person : 421
  • fire : 623

all my dataset (train / validation) from Fsimo dataset

i used yolov3.cfg with the following configuration :

• width & height : 416
• batch : 64
• subdivision : 16
• learning rate : 0.001
• max batches : 60000
• steps : 48000, 54000
• i trained for 8000 iteration

and i already followed all these rules how to improve object detection

i know that i should have at least 2000 image for each class , but i just want to check if i am on the right path.

i reduced the person images compared to fire images in the training dataset , because the models i used for training they already extracted features for person

i tried two paths :

1. transfer learning using darknet53.conv.74 : the best mAP i got is 56%

   net.optimized_memory = 0 
   batch = 1, time_steps = 1, train = 0 
   
   seen 64, trained: 262 K-images (4 Kilo-batches_64) 
   
   calculation mAP (mean average precision)...
   
   detections_count = 5133, unique_truth_count = 1040  
   class_id = 0, name = Person, ap = 59.97%     (TP = 254, FP = 176) 
   class_id = 1, name = Fire, ap = 52.11%       (TP = 327, FP = 219) 
   
   for conf_thresh = 0.25, precision = 0.60, recall = 0.56, F1-score = 0.58 
   for conf_thresh = 0.25, TP = 581, FP = 395, FN = 459, average IoU = 41.90 % 
   
   IoU threshold = 50 %, used Area-Under-Curve for each unique Recall 
   mean average precision (mAP@0.50) = 0.560427, or 56.04 % 

2. fine tuning using yolov3.conv.81 : the best mAP i got is 57%

   net.optimized_memory = 0 
   batch = 1, time_steps = 1, train = 0 
   
   seen 64, trained: 483 K-images (7 Kilo-batches_64) 
   
   calculation mAP (mean average precision)...
   
   detections_count = 4112, unique_truth_count = 1040  
   class_id = 0, name = Person, ap = 70.14%     (TP = 289, FP = 165) 
   class_id = 1, name = Fire, ap = 44.94%       (TP = 255, FP = 168) 
   
   for conf_thresh = 0.25, precision = 0.62, recall = 0.52, F1-score = 0.57 
   for conf_thresh = 0.25, TP = 544, FP = 333, FN = 496, average IoU = 45.30 % 
   
   IoU threshold = 50 %, used Area-Under-Curve for each unique Recall 
   mean average precision (mAP@0.50) = 0.575397, or 57.54 % 

my questions are :

what i can do to increase the mAP to 80 - 90 mAP ?

is it possible to reach 80 - 90 mAP with yolov3 , since the best mAP YOLOv3-416 reach on coco is 55.3% mAP-50 ?

what path should i continue with transfer learning or fine tuning ?

is there any other configurations i have to do ?

and if the answer is to add more images , how many images i have to add to increase the mAP to 80-90 mAP ?

should the person ( images / objects ) =~ fire ( images / objects ) in both training and validation sets ?

Fsimo dataset has both fire and smoke images , i considered both fire and smoke as one object , is it a good idea ?

i will appreciate any help : )

[isra60]

Do you have a link to the Fsimo dataset?

[ahmadalzoubi13579]

@isra60 https://github.com/mtcazzolato/dsw2017

[lsd1994]

Why not try yolov4?

[ghost]

check that https://github.com/amineHY/YOLOv3-for-custum-objects

[ahmadalzoubi13579]

@isra60 i will try it soon.