Yolov3-Tiny vs Yolov3-Tiny-Xnor [Comparison]

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )

http://pjreddie.com/darknet/

Other

21.65k stars 7.96k forks source link

Yolov3-Tiny vs Yolov3-Tiny-Xnor [Comparison] #2605

Closed dexception closed 5 years ago

dexception commented 5 years ago

Yolov3-Tiny-XNOR After 2K Iterations

After 8K Iterations

After 10k iterations

Yolov3-Tiny After 1,76,000 Iterations

I have a couple of queries:

The bounding boxes of XNOR model don't seem to be accurate enough.
The performance improvement is only marginal on recorded video 40 FPS vs 50 FPS on the same gpu.

Thanks.

AlexeyAB commented 5 years ago

@dexception Hi,

Yolov3-Tiny-XNOR After 10k iterations

Yolov3-Tiny After 1,76,000 Iterations

Did you train Yolov3-Tiny-XNOR - 10k iterations and train Yolov3-Tiny 176k iterations?
What result do you get after training Yolov3-Tiny-XNOR the same 176k iterations?
What GPU do you use?
yolov3-tiny is too small, so there is a bottleneck on the 1st and last layers that should be FP32. Also on modern GPUs yolov3-tiny too small so there is added bottleneck on CPU. So on modern GPU better to use yolov3-spp_xnor_obj.cfg https://github.com/AlexeyAB/darknet/issues/2365#issuecomment-462923756 with this pre-trained weights file: https://drive.google.com/open?id=1IT-vvyxRLlxY5g9rJp_G2U3TXYphjBv8

dexception commented 5 years ago

Xnor was giving me plenty of false detections. So i stopped the training after 10K iterations and added an additional 40K negative dataset.
I will share the results of Yolov3-Tiny-XNor with 40 K negative images after 176K iterations. It is still running.
Training GPU is 1080 TI. Inference on my laptop Nvidia 940 MX
Thanks for this info. It is for this reason(High CPU) i not been able to scale the solution and forced to use other detectors. Let me retrain this on my current dataset and will post the results.

dexception commented 5 years ago

Xnor Output after 176K iterations with 40K Negative Dataset. As you can see it is not Accurate in complex situations.

Will post a day after tomorrow about yolov3-spp_xnor_obj.cfg.

PS: Can you make the title of the bounding box transparent ?

AlexeyAB commented 5 years ago

@dexception

Will post a day after tomorrow about yolov3-spp_xnor_obj.cfg.

Try to train yolov3-spp_xnor_obj.cfg. with this exactly this darknet53_448_xnor.conv.74 pre-trained weights: https://drive.google.com/open?id=1IT-vvyxRLlxY5g9rJp_G2U3TXYphjBv8

dexception commented 5 years ago

I will retrain with the new file and get back to you in a couple of days.

dexception commented 5 years ago

@AlexeyAB After training for about 50K iterations. The detection is definitely better but the bounding boxes are all over the place. I am getting better intersection accuracy with yolov3-tiny then xnor variants.

The only issue with yolov3-tiny is that it is always consuming more than 2 cores Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz which means that i will have to have scalable xeons on machine which increases the cost of the solution 5 times.

AlexeyAB commented 5 years ago

@dexception

After training for about 50K iterations. The detection is definitely better but the bounding boxes are all over the place. I am getting better intersection accuracy with yolov3-tiny then xnor variants.

Do you mean that yolov3-spp_xnor_obj.cfg has more True Positives, byt yolov3-tiny.cfg has higher IoU? What mAP do you have for yolov3-spp_xnor_obj.cfg, yolov3-tiny-xnor.cfg and yolov3-tiny.cfg ?

The only issue with yolov3-tiny is that it is always consuming more than 2 cores Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz which means that i will have to have scalable xeons on machine which increases the cost of the solution 5 times.

Do you use yolov3-tiny.cfg on CPU instead of GPU?

In your case with many small object (faces) may be it's better to train the model with higher resolution width=832 height=832 in cfg-file or 1024x1024.

dexception commented 5 years ago

yolov3-tiny.cfg@416 is giving me better accuracy both in terms of detection and correct intersection than yolov3-spp_xnor_obj.cfg.

I am ignoring the map results and instead verifying the output manually on the test dataset of 100 images because IOU accuracy is more important to me.

I have trained yolov3-tiny.cfg on the exact same GPU machine. I don't know the internals of how xnor is working but the results are not good.

If you can release the C++ wrapper for yolo2_light i would like to compare the TensorRT performance with yolo2_light of yolov3-tiny vs OpenVino.

AlexeyAB commented 5 years ago

@dexception Can you show example of detection by using yolov3-spp_xnor_obj.cfg on your image?

dexception commented 5 years ago

The dataset was tagged to not include side faces. Yolov3-tiny is giving me better output for the same resolution.

dexception commented 5 years ago

Can you share your plans to release the c++ wrapper for yolo2_light ?

AlexeyAB commented 5 years ago

Do you mean C++ API for DLL/SO library that is based on yolo2_light?

Currently there is no plans to add C++ wrapper for yolo2_light. May be I just will make it possible to compile yolo2_light in both C and C++, so you wil can add these files to your code - to built-in it to your C/C++ code.

dexception commented 5 years ago

@AlexeyAB Any update on how can i use yolo2_light with c++ ?