Loss is plotted but not mAP

[dalhorde]

Hi everyone,

I'm very new in Yolo and Object Detection and I have a problem somewhere. I am trying to detect my own objects with a docker container. I pulled inqdocker/docker on dockerhub. However I followed all the steps explained by AlexeyAB on https://github.com/AlexeyAB/darknet#how-to-train-to-detect-your-custom-objects , I do not have mAP plotted when I start my simulation. I can't see the reason why. I have checked the annotation and I do not think it is the problem. It may comes from my file locations but I'm not sure.

My goal is to do object detection with ships on satellite images so I only have one class to detect. I have 24153 pictures for my test database and 168403 pictures for my training (it may be too much I don't know) Here is my file's contents:

obj.data :

classes = 1
bateau = data/bateau.txt
valid = data/test.txt
names = obj.names
backup = backup 

obj.names : bateau

yolov4-obj.cfg :

yolov4-obj.txt

This is my tree structure :

.
|-- generation_train.py
|-- generation_test.py
|-- yolov4.conv.137
|-- obj.names
|-- 3rdparty
|   |-- pthreads
|   |   |-- bin
|   |   |-- include
|   |   `-- lib
|   `-- stb
|       `-- include
|-- backup
|-- build
|   `-- darknet
|       `-- x64
|           |-- backup
|           |-- cfg
|           |-- data
|           |   |-- labels
|           |   `-- voc
|           `-- results
|-- cfg
|   `-- yolov4-obj.cfg
|-- cmake
|   `-- Modules
|-- data
|    |-- obj.data
|    |-- labels
|    |-- obj
|         |-- ALL MY .JPG TRAINING PICTURES
|         `-- ALL MY .TXT FILE
|    |-- test
|         |-- ALL MY .JPG TEST PICTURES
|         `-- ALL MY .TXT FILE       
|-- from_host
|-- include
|-- obj
|-- results
|-- scripts
|   |-- log_parser
|   `-- windows
`-- src

I have downloaded the yolov4.conv.137 file on github and I am using it. I have downloaded generate_train.py and generate_test.py and I am using it

My final command line is : ./darknet detector train data/obj.data cfg/yolov4-obj.cfg yolov4.conv.137 -map

During the execution I have this line :

(next mAP calculation at 12034 iterations)

I found it strange because I thought mAP is calculated every 1000 iterations

At the end of the execution, I have this :

train: Using default 'data/train.txt'

 calculation mAP (mean average precision)...
24156
 detections_count = 8009, unique_truth_count = 10481  
class_id = 0, name = bateau, ap = 0.00%      (TP = 0, FP = 729) 

 for conf_thresh = 0.25, precision = 0.00, recall = 0.00, F1-score = -nan 
 for conf_thresh = 0.25, TP = 0, FP = 729, FN = 10481, average IoU = 0.00 % 

 IoU threshold = 50 %, used Area-Under-Curve for each unique Recall 
 mean average precision (mAP@0.50) = 0.000000, or 0.00 % 
Total Detection Time: 184 Seconds

Set -points flag:
 `-points 101` for MS COCO 
 `-points 11` for PascalVOC 2007 (uncomment `difficult` in voc.data) 
 `-points 0` (AUC) for ImageNet, PascalVOC 2010-2012, your custom dataset

 mean_average_precision (mAP@0.5) = 0.000000 
New best mAP!
Saving weights to backup/yolov4-obj_best.weights
Saving weights to backup/yolov4-obj_6000.weights
Saving weights to backup/yolov4-obj_last.weights
Saving weights to backup/yolov4-obj_final.weights

If you have any ideas, they are most welcome.

[sarwasg]

Can you show the name of image files? I have similar problem, when I use string in the name of image file. After saved detected instances in coco style json file all detections have image_id=0.

[dalhorde]

Can you show the name of image files? I have similar problem, when I use string in the name of image file. After saved detected instances in coco style json file all detections have image_id=0.

b7e7701c6.jpg
b7e7701c6.txt

e80caab37.jpg
e80caab37.txt

My files look like this. For each image name, I have the same txt file. In addition, in my train.txt file I have this kind of content :

data/obj/aa04a51b7.jpg
data/obj/702fc3053.jpg
data/obj/d9c5dfe42.jpg

[dalhorde]

you have to pass the map flag along with the train command

./darknet detector train data/obj.data cfg/yolov4-obj.cfg yolov4.conv.137 -dont_show -map

from next time onward please attach your .cfg file with a .txt extension, pasting it here reduces the readability of the code.

Yes I did put the -map flag in the command line.

[rickymedrano]

@dalhorde It's not showing mAP on the graph because its 0 the whole time. The reason why is that you messed up the cfg file, you put ALL filters to number of 18. From front page

keep in mind that it only has to be the last [convolutional] before each of the [yolo] layers.

So go back and change filters=18 for only the convolutional layers before each yolo layer, there will be 3 changes. All other filters need to go back to default so you might want to download a new copy of yolo4.cfg then make the changes. Or you can use mine which I used to train 1 class just like you. See attached. Might need to change subdivisions depending on your GPU

yolov4-custom.cfg.txt