model is only detecting objects from training dataset.

[AdelBits]

Hello ,

I am training yolov3 using google colab to obtain a model that can detect pistols. Dataset is 2500 images and stopping around 5000 iterations and the resulting model correctly detected objects when testing on an image from input dataset.

I read more in the issues section before posting this , I saw something about a validation set and that this set must be a secondary dataset that differs from the train dataset. currently my test.txt is similar to train.txt (I could not get a clear idea on how to set a validation set from the docs).

I trained numerous times , even on 9000 iterations where avg loss barely changes and tried earlier weights but the result obtained was overfitting it seems. how can I escape that issue and obtain a model that can generalize far more than the training dataset?

thanks to @AlexeyAB and the community for the huge efforts on this framework.

[AlexeyAB]

• Set valid=train.txt in obj.data file. What mAP can you get?
• What cfg-file do you use?

https://github.com/AlexeyAB/darknet#how-to-improve-object-detection

for each object which you want to detect - there must be at least 1 similar object in the Training dataset with about the same: shape, side of object, relative size, angle of rotation, tilt, illumination. So desirable that your training dataset include images with objects at diffrent: scales, rotations, lightings, from different sides, on different backgrounds - you should preferably have 2000 different images for each class or more, and you should train 2000*classes iterations or more

[AdelBits]

ty @AlexeyAB for your precious feedback. the valid = train.txt currently and getting 0.17 avg loss , should I retrain and use -map command at the end to check mAp?

I am using yolo.cfg and modifying it according to the documentation.

the dataset contains a lot of variations for the pleased object.

[AlexeyAB]

should I retrain and use -map command at the end to check mAp?

no. run https://github.com/AlexeyAB/darknet#when-should-i-stop-training ./darknet detector map obj.data obj.cfg obj.weights

[AdelBits]

thanks again for feedback @AlexeyAB , i will test again and report back with results

[AdelBits]

Hello again @AlexeyAB thanks for assist and efforts. Back with some results.

at 9136th iteration the highest mAp was obtained and it was 73.42% and it gave me the best weights file. I did not continue training because afterwards mAp was decreasing.

avg loss landed on ~0.62

the model now can detect objects in different pics away from the dataset , but was not able to detect objects in some ; at least there is a result.

In videos , detection seems glitchy. The model detects the object correctly at some frames and in some frames it does not detect it anymore and/or it detects some other objects .and i will post an example picture.

A frame showing the model correctly detecting the object.

In this frame the model is mistaking the object for another one

how to solve is this issue of losing/blinking detection and some times mistaking detections , in such a way to make the detection more accurate and realistic?

[AlexeyAB]

• Check your dataset by using Yolo_mark, are there all boxes correct?

• What cfg-file did you use for training?

[AdelBits]

Thanks for your fast response @AlexeyAB and sorry for taking some off your valued time

is there a version for yolo_mark on mac? and i am using yolov3-tiny.conv.15

[AlexeyAB]

@falconitig

Firstly try to run training with flag -show_imgs at the end of training command, do you see correct truth bounded boxes?

• Yolo_mark can be compiled on Mac too, if there is OpenCv installed: https://github.com/AlexeyAB/Yolo_mark

• Or you can try to use another tool for checking dataset: https://github.com/AlexeyAB/darknet#how-to-mark-bounded-boxes-of-objects-and-create-annotation-files

and i am using yolov3-tiny.conv.15

Show your cfg-file, not weights-file. And what training command do you use?

[AdelBits]

thanks again sir @AlexeyAB

I used google colab for training and it didnt let me to use -show_imgs command

and I double-checked the dataset with the LabelImg tool which was used for drawing the boxes , everything seems fine ; all objects are labeled the way they should be seen.

I apologise for cfg mistake , yolov3-tiny-obj.cfg was used and here is it :

[net] Testing batch=64 subdivisions=2 Training batch=64 subdivisions=2 width=416 height=416 channels=3 momentum=0.9 decay=0.0005 angle=0 saturation = 1.5 exposure = 1.5 hue=.1

learning_rate=0.001 burn_in=1000 max_batches = 500200 policy=steps steps=400000,450000 scales=.1,.1

[convolutional] batch_normalize=1 filters=16 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=2

[convolutional] batch_normalize=1 filters=32 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=2

[convolutional] batch_normalize=1 filters=64 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=2

[convolutional] batch_normalize=1 filters=128 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=2

[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=2

[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=leaky

[maxpool] size=2 stride=1

[convolutional] batch_normalize=1 filters=1024 size=3 stride=1 pad=1 activation=leaky

###########

[convolutional] batch_normalize=1 filters=256 size=1 stride=1 pad=1 activation=leaky

[convolutional] batch_normalize=1 filters=512 size=3 stride=1 pad=1 activation=leaky

[convolutional] size=1 stride=1 pad=1 filters=18 activation=linear

[yolo] mask = 3,4,5 anchors= 1.48,1.66, 3.42,3.11, 3.56,6.85, 7.67,5.24, 7.81,9.21, 11.74,10.86 classes=1 num=6 jitter=.3 ignore_thresh = .7 truth_thresh = 1 random=1

[route] layers = -4

[convolutional] batch_normalize=1 filters=128 size=1 stride=1 pad=1 activation=leaky

[upsample] stride=2

[route] layers = -1, 8

[convolutional] batch_normalize=1 filters=256 size=3 stride=1 pad=1 activation=leaky

[convolutional] size=1 stride=1 pad=1 filters=18 activation=linear

[yolo] mask = 0,1,2 anchors = 1.48,1.66, 3.42,3.11, 3.56,6.85, 7.67,5.24, 7.81,9.21, 11.74,10.86 classes=1 num=6 jitter=.3 ignore_thresh = .7 truth_thresh = 1 random=1

[AlexeyAB]

comment these 2 lines: https://github.com/AlexeyAB/darknet/blob/0d98f20be1290d2a0e495577572893854f84607e/src/data.c#L975-L976

and these 2 lines: https://github.com/AlexeyAB/darknet/blob/0d98f20be1290d2a0e495577572893854f84607e/src/data.c#L1143-L1144

recompile

run with -show_imgs and look at created files like aug_ .... .jpg

[AdelBits]

hello @AlexeyAB I commented the lines recompiled again and attempted to run it , it greeted me with the same error.

Furthermore , I checked the bounding boxes using yolo_mark today , all boxes correctly shown. surprisingly i found that my dataset has a picture of the man involved in the video :

Suggestion: should i upload my files to dropbox?

[AlexeyAB]

So your labels are correct. It's good.

Dataset is 2500 images

• So do you have 2500 images in Training dataset?
• Do you have Validation dataset? https://github.com/AlexeyAB/darknet#when-should-i-stop-training
• What mAP can you get on Training dataset?
• What mAP can you get on Validation dataset?

---

Try to follow this instruction and run training with -dont_show flag

[AdelBits]

1800 now. I had 2500 in the first attempt , but then I decided to try a new set and reset everything to start fresh. so 1800 for training and test set is 200 , valid = test.txt

-__- (labeling the images is a very tedious task)

I ran : !./darknet detector map data_for_colab/obj.data data_for_colab/yolov3-tiny-obj.cfg data_for_colab/Backup/yolov3.weights

this weights file (9000 iterations) gave me the highest mAp , while at 8000 iterations weights file gave me 62.24 mAp

@AlexeyAB thanks for your support and sorry from taking off your time, what I am working on here is a very serious project, a vision of mine and I want to implement on reality as the project has alot of aspects and categories yet to be developed, and this feature here is the start. thanking you is not enough.. and hopefully, we can get this fixed to obtain the best results.