mAP, Precision, and Recall All Zero When Using DetectNet

[pinsondg]

I am trying to train an image detection model to detect potholes in the road. I have around 1700 training images, all 640 x 640 and 140 validation images of the same size. I have trained the model around 10 times with different settings described in other posts but still am having no luck getting the mAP, precision, and recall above zero. I also ran it for 300 Epochs overnight and still nothing.

Here is some of the other things I have tried: https://github.com/NVIDIA/DIGITS/issues/1108

Here is the latest setup I used to train:

We are also using bvlc_googlenet.caffemodel for our pre-trained model.

The labels are all in KITTI format and here is an example of one label: pothole 0.0 0 0.0 362.4076 237.5474 372.146 242.4166 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Here is an example of an image, all of them are shot from a dashboard cam:

[akTwelve]

Edit: I got mine working. My problem was that I did not set the Custom Classes field correctly when I created the dataset. In your case, it should be "dontcare, pothole". I originally left out the "dontcare" class, so I think it was ignoring my class.

Original comment: I'm having the same issue with a dataset I created of 512x512 png images and KITTI style txt files. Very interested to know if you get it resolved. I'll also report back if I figure it out.

[pinsondg]

I resolved my problem it turns out the dataset we were using was not very good and the objects we were trying to detect were too small. Once we got new images and trained it again it worked well.

[luhongwei]

I also faced the mAP=0 probelm. I pulled about 1000 pictures with boats from MS-COCO data set in order to train a model that recognizes boats. But on my DIGITS, after about 300 epochs, the conv1_7x7_s2 became so "pale" like in attached picture even through I had started training from a pre-trained model, detectnet(KITTI)

I'm wondering if the training set was not good enough since I put small boats pictures in a generated 1024x768 background. like in attached

By the way, I did use Alps Labeling Tool to check the correctness of labels.

[dmitry-vlt]

did you resolve your issue? I am facing a similar problem

[lu-hongwei]

I just re-installed DIGITS and it's gone. now mAP>0.

Also here is a tick that might help: remove those less-than-16-pixels-bboxs from your training set;

[MaxKinny]

Edit: I got mine working. My problem was that I did not set the Custom Classes field correctly when I created the dataset. In your case, it should be "dontcare, pothole". I originally left out the "dontcare" class, so I think it was ignoring my class.

Original comment: I'm having the same issue with a dataset I created of 512x512 png images and KITTI style txt files. Very interested to know if you get it resolved. I'll also report back if I figure it out.

This solve my problem! THX!