Help with false positives

[Voicech]

Hello,

I have trained yolo v3 tiny with two classes: for detecting bicycles and pedestrians. Training size was around 4000 pictures for each class, trained for around 20000 iterations.

In certain cases yolo fails giving me a false positive (see picture attached). Surprisingly, it fails on the background image, and the prediction of that fail positive is 90%. To mention - I was training using the pictures from the same camera, with the same background, just annotating bicycles/pedestrians on it.

How to deal with such issues - create a third class and name it "fails"? Also, I have issues with snow cleaning machines and kid trolleys, which sometimes are detected as bicycles. What's the best approach to avoid such cases? Creating a separate class for each type of a fail? I considered one approach mentioned somewhere in the forum, by adding false positives to the training set with empty annotation files. I tried it but cannot say it helped.

Any help much appreciated.

[AlexeyAB]

@Voicech Hi,

In certain cases yolo fails giving me a false positive (see picture attached). Surprisingly, it fails on the background image, and the prediction of that fail positive is 90%. To mention - I was training using the pictures from the same camera, with the same background, just annotating bicycles/pedestrians on it.

• Is this image from Training dataset?

• What mAP, TP/FP/FN, Precision-Recall - do you get on Training (and if you have Validation) datasets? https://github.com/AlexeyAB/darknet#when-should-i-stop-training ./darknet detector map data/obj.data yolo-obj.cfg backup\yolo-obj_20000.weights

• Do you get this image by stitching several images?

• Did you label your training dataset by using Yolo_mark? https://github.com/AlexeyAB/Yolo_mark

• Do you use training images with overlays "Camera 01", "Person 1", "06-05-1970 ..." ?

---

How to deal with such issues - create a third class and name it "fails"? Also, I have issues with snow cleaning machines and kid trolleys, which sometimes are detected as bicycles. What's the best approach to avoid such cases? Creating a separate class for each type of a fail? I considered one approach mentioned somewhere in the forum, by adding false positives to the training set with empty annotation files. I tried it but cannot say it helped.

The best practice is to use negative samples, with images without objects and empty txt-files, so in total you should have 4000 images with objects + 4000 images without objects. Check that in the train.txt there are 8000 images: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

desirable that your training dataset include images with non-labeled objects that you do not want to detect - negative samples without bounded box (empty .txt files) - use as many images of negative samples as there are images with objects

[Voicech]

Hi Alex,

Is this image from Training dataset?

No, this is image from production implementation

What mAP, TP/FP/FN, Precision-Recall - do you get on Training (and if you have Validation) datasets?

Don't have it saved, but it was pretty good based on your instructions

Do you get this image by stitching several images?

No, this is a screenshot from the camera. Yolo Tiny on Jetson TX2 is capable of roughly 6-7 fps. I have implemented the feature "save screenshot on detection", so this is the source of the picture

Did you label your training dataset by using Yolo_mark? https://github.com/AlexeyAB/Yolo_mark

Nope, I used https://github.com/tzutalin/labelImg , but I believe the output should be the same (attached)

Do you use training images with overlays "Camera 01", "Person 1", "06-05-1970 ..." ?

Yes, but usually it doesn't have impact on the detection accuracy. I get fails in very specific conditions (e.g. snow, or some very unusual objects in sight)

Finally, i didn't notice that I need to feed as much negative samples as the dataset size. I will do that on next training.

Repeating my question - what to do with trolleys, snow machines, etc.? Feed as negative samples or create a separate class for them?

vlcsnap-error993 1 (1).txt

[AlexeyAB]

Repeating my question - what to do with trolleys, snow machines, etc.? Feed as negative samples or create a separate class for them?

Feed as negative samples. As described here: https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

Your training dataset should be similar to the test (production) images as much as possible.

Also check all your images by using Yolo_mark.

"Camera 01", "Person 1", "06-05-1970 ..." - it can affect on false negatives, if you doesn't provide sufficient negative samples with similar "Camera 01", "Person 1", "06-05-1970 ..."

This is bad detection, this should happen extremely rarely.

[Voicech]

Hi again,

I a preparing for my new training, and the question is: you could see my production environment in the pictures above.

Previously I was training on the samples collected from that camera, and marking a label strictly around the bicycle. However, as you could see in the example below, I must label also some parts of the rider (especially legs). As a result, I also get false positives while pedestrians passing by, as Yolo sometimes sees bicycle in their legs.

Is it better to use super clean pictures for training like in the picture below? What about the background of such picture? Is it important to train with the real background? Help much appreciated.

[AlexeyAB]

Hi,

Is it better to use super clean pictures for training like in the picture below?

Better to train with original images from a camera (not super clean). To avoid false-positives, just include images with false-positives to your Training dataset (without labels on false-positive objects).

What about the background of such picture? Is it important to train with the real background?

Yes.

[Voicech]

Thanks so much!

On Fri, Jan 25, 2019 at 11:36 AM Alexey notifications@github.com wrote:

Hi,

Is it better to use super clean pictures for training like in the picture below?

Better to train with original images from a camera (not super clean). To avoid false-positives, just include images with false-positives to your Training dataset (without labels on false-positive objects).

What about the background of such picture? Is it important to train with the real background?

Yes.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AlexeyAB/darknet/issues/2212#issuecomment-457512872, or mute the thread https://github.com/notifications/unsubscribe-auth/AsjaVdRdqVzFes-De-ygSmJCoMfGnFpIks5vGtARgaJpZM4aBKOa .

[litingsjj]

@AlexeyAB Sorry to bother you! I have questions about false-positive. When I found some false-positive in some images with finished model. I put images with false-positive(without labels on false-positive objects) to training dataset. Here I have two questions:

1. About images number with false-poisitive, if I detect only person, now I detect bicycle as person, Am I put images include bicycle as the same number as train dataset? If I detect others as person again, what should I do? Put images include other thing again? But now the the images with false-positive is larger than training dataset. If I reduce number of images include bicycle, is it influence false detect bicycle?

2. Am I need to retrain network or finetune origin model? If finetune, how to set parameters about lr, iteration or something else?

I'm appreciate if you have time to give me some advice!