How to detect smaller objects with custom object datasets

[praneeth0609]

Hello @AlexeyAB, first of all, I want to thank you for the great works and now I need your suggestion. Currently, I'm working on custom object detection like birds, etc. I need to detect small objects also. but am not able to detect small objects. To get detect small objects, what I need to do. Here are my questions:

1. How many images required per class ? (am using 2 classes.)
2. For smaller object detection what I have to do ?
3. even the object is very small in the frame. it should be detected . for this any suggestions?
4. If we take one bird as an example, how many images required for specific angle of bird(front, back, up, bottom) ?
5. what is the minimum object size that i can detect using yolo. ?
6. For smaller object detections, what changes I have to do to my custom object dataset ?
7. Any suggestions for improving the mAP ?

[Farjad3253]

@praneeth0609 I am also working currently on a project to detect objects which are even smaller than mentioned by you. The objects are small stones, nuts, bolts, screws etc. I am also facing the same issue that I have trained Yolov3 and Yolov4 on my custom dataset but not able to get good results. So any solution by anybody?

[dsbyprateekg]

@Farjad3253 @praneeth0609 have you trained the dataset following below steps as described by the author?

[stephanecharette]

@Farjad3253 @praneeth0609 have you trained the dataset following below steps as described by the author?

Which doesn't apply to the tiny or tiny-3l configurations, so I couldn't use those instructions. :( See issue #6274.

I did test out YOLOv3-tiny, YOLOv3-tiny-3l, YOLOv4-tiny, and YOLOv4-tiny-3l and then post a video to compare the results of each one: https://www.youtube.com/watch?v=p0Wn8ZNQ_uc

The problem is YOLOv4-tiny-3l is already at the edge of what the Jetson Nano can process in realtime for what I'm doing.

[dsbyprateekg]

@stephanecharette have you tested the small objects changes with tiny yolo?

[stephanecharette]

@stephanecharette have you tested the small objects changes with tiny yolo?

Did you see issues #6274?

[dsbyprateekg]

@stephanecharette Yes I saw but there you have not mentioned that you have trained your model with those changes so I asked.

[stephanecharette]

What changes? The ones you mentioned are for YOLOv4. Not YOLOv4-tiny, nor YOLOv4-tiny-3l. If you can show me the changes to make to YOLOv4-tiny or YOLOv4-tiny-3l, then I'll try them. That is what I was asking in issue #6274.

[dsbyprateekg]

@stephanecharette It's my bad. I misunderstood you. As I undstood now you have applied below changes and trained your tiny yolo model but you are not able to detect all objects?

Please correct me if I misunderstand again.

[stephanecharette]

That is incorrect. Those changes cannot be applied to YOLOv4-Tiny nor to YOLOv4-Tiny-3l. The changes you keep quoting are only for the full YOLOv4.

What I would like is the equivalent changes but for YOLOv4-Tiny and YOLOv4-Tiny-3l.

[dsbyprateekg]

@stephanecharette I do not find any statement in the repowhich says that thsoe changes are only for YOLO and not for Tiny YOLO so I kept quote. It will be better to get confirmation from the author @AlexeyAB .

[stephanecharette]

@stephanecharette I do not find any statement in the repowhich says that thsoe changes are only for YOLO and not for Tiny YOLO so I kept quote. It will be better to get confirmation from the author @AlexeyAB .

The problem is the lines it says to modify, and the values it says to modify, don't exist in Tiny nor Tiny-3L. So I have no idea what to change.

[Farjad3253]

@stephanecharette , @dsbyprateekg . I have tried the model YOLOv4 with the modifications as specified by AlexeyAB. i.e. as mentioned below:

But I am still not able to get results in smaller objects like screws, bolts, nuts, stones etc. So now any way out as I want to solve this issue?

[stephanecharette]

We'd have to know more about how you trained, how you made sure your labels are correct, the number of iterations you trained for, the exact commands you ran, etc. Just saying "not able to get results" isn't enough for us to guess. Can you share your project? E.g., some of your images and labels, along with your .cfg file?

[praneeth0609]

Hello @AlexeyAB, first of all, I want to thank you for the great works and now I need your suggestion. Currently, I'm working on custom object detection like birds, etc. I need to detect small objects also. but am not able to detect small objects. To get detect small objects, what I need to do. Here are my questions:

1. How many images required per class ? (am using 2 classes.)
2. For smaller object detection what I have to do ?
3. even the object is very small in the frame. it should be detected . for this any suggestions?
4. If we take one bird as an example, how many images required for specific angle of drone(front, back, up, bottom) ?
5. what is the minimum object size that i can detect using yolo. ?
6. For smaller object detections, what changes I have to do to my custom object dataset ?
7. Any suggestions for improving the mAP ?

can anyone clarify my questions....?

[praneeth0609]

hello @AlexeyAB @stephanecharette I have a camera with 30X optical Zoom, and I want to detect birds, which are at long distances. give me some suggestions:

for the above scenario which type of images I have to take for training. a. high resolution or low resolution. b. what should be the bird size in training images(custom dataset). I'm using yolo object detection algorithm. please give me some suggestions.

[stephanecharette]

The training/validation pictures should be exactly like you ones you want to detect. The birds should take up exactly the same size in the image as the ones you want to detect. If you train with close-up images, then the network will only detect close-up images. Whatever you want to detect, that is what should be in your training/validation images.

[stephanecharette]

can anyone clarify my questions....?

You didn't answer my questions in the previous comment, so it is hard for anyone to answer you.

[praneeth0609]

The training/validation pictures should be exactly like you ones you want to detect. The birds should take up exactly the same size in the image as the ones you want to detect. If you train with close-up images, then the network will only detect close-up images. Whatever you want to detect, that is what should be in your training/validation images.

@stephanecharette thank you for your replay... I will test with your inputs, and I will update you soon.

[Farjad3253]

We'd have to know more about how you trained, how you made sure your labels are correct, the number of iterations you trained for, the exact commands you ran, etc. Just saying "not able to get results" isn't enough for us to guess. Can you share your project? E.g., some of your images and labels, along with your .cfg file?

Yes @stephanecharette . Definitely I will share all the necessary files.

Below I have attached a zip file containing some of my dataset images along with annotations txt files in yolo format labelled : data-yolo-fomat.zip

[Farjad3253]

@stephanecharette . Sir furthermore I trained for 6000 iterations on 1004 images belonging to a single class i.e. FOD (Foreign Object Debris). The config file is as attached below:

yolov4-obj-backup.zip

[stephanecharette]

I took a peek at the images, markup, and configuration. Your configuration is setup to resize the images to 608x608. But when you have images like this:

and

The objects you are trying to find are approximately 5x5 pixels in size after the image is resized to 608x608. This is too small to reliably detect.

The rest of the images looks great. I'm certain that with enough images and max_batches, this would result in a great neural network:

What does chart.png look like once you remove the images with too-small objects and re-train?

Note that 6000 for max_batches is a good start, but you'll probably need to eventually increase that. See here for example: https://www.ccoderun.ca/programming/2020-01-04_neural_network_training/

[Farjad3253]

@stephanecharette . Thanks Sir for helping me out. Sir the chart.png rapidly goes down at the start and after some iterations it slowly decreases. But sir the main issue is that I have more images in my dataset in which the objects are very small like the above image which you also have highlighted. Now sir what is the solution if I want these type of small objects to get detected?

[praneeth0609]

The training/validation pictures should be exactly like you ones you want to detect. The birds should take up exactly the same size in the image as the ones you want to detect. If you train with close-up images, then the network will only detect close-up images. Whatever you want to detect, that is what should be in your training/validation images.

@stephanecharette I want some more information regarding this. for example my image resolution is 1920x1080 , in that image my object size is 16 x16 pixels, i labeled the 16x16 pixels object in my 1920x1080 image. is it enough or the full image should be 16x16... please clarify this. thank you.

[stephanecharette]

If your image is 1920x1080, and the object is 16x16, then once the image is resized to 416x416 (YOLOv4-tiny default), that means your object is around 3x6 pixels. This is why you cannot detect it.

[praneeth0609]

If your image is 1920x1080, and the object is 16x16, then once the image is resized to 416x416 (YOLOv4-tiny default), that means your object is around 3x6 pixels. This is why you cannot detect it.

@stephanecharette thank you for your reply. I'm training custom dateset, with yolov4(not yolov4-tiny). am able to train with 416x416 width and height. but for smaller object detection I need to use 608x608. while using 608x608 width and height in yolov4.cfg am getting CUDA error.

**error: Create 6 permanent cpu-threads Try to set subdivisions=64 in your cfg-file. CUDA status Error: file: /home/abc/cmake-3.18.1/darknet/src/dark_cuda.c : () : line: 373 : build time: Aug 8 2020 - 16:58:31

CUDA Error: out of memory CUDA Error: out of memory: Resource temporarily unavailable Segmentation fault (core dumped)**

it is showing try to set subdivisions = 64, but my subdivisions already 64. please help me on this. am attaching my cfg and error message screenshot.here.

system properties: cpu : i7 gpu: GeForce RTX 2060 (6 GB) platform: Linux (UBUNTU) Thank you yolo4.zip

[Anafeyka]

@praneeth0609 You can't train Yolo 4 with 6 GB of GPU Memory you need to change the random parameter to 0 https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov4-custom.cfg#L1151

[praneeth0609]

@Anafeyka Thank you for your suggestion. but I'm able to train yolov4 with 416x416 height and width (in .cfg) file. if change 416x416 to 608x608, it is showing like the error mentioned in the above comment. can you tell me, what is the minimum GPU memory required to train yoloV4. if i want to keep random parameter to 1.

[Anafeyka]

@praneeth0609 Theoretically, 8gb 2070/1070 should be enough. I'm not sure about this because I only have a 6/12/24 gb gpu.

[praneeth0609]

@Anafeyka thank you for your replay. if we train yolov4 with random = 0; can we able to detect smaller objects..? did you tried it..?

[Anafeyka]

@praneeth0609 The random parameter switches resolution every 10 iterations during training. alternatively, you can use the resize parameter instead of random. Example

random=0
resize=1.5

To detect small objects, you must modify your cfg.

for training for small objects (smaller than 16x16 after the image is resized to 416x416) - set layers = 23 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L895

set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L892 set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L989

[praneeth0609]

@praneeth0609 The random parameter switches resolution every 10 iterations during training. alternatively, you can use the resize parameter instead of random. Example

random=0
resize=1.5

To detect small objects, you must modify your cfg.

for training for small objects (smaller than 16x16 after the image is resized to 416x416) - set layers = 23 instead of

https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L895

set stride=4 instead of

https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L892

set stride=4 instead of https://github.com/AlexeyAB/darknet/blob/6f718c257815a984253346bba8fb7aa756c55090/cfg/yolov4.cfg#L989

@Anafeyka thank you for your replay. and I want some more information regarding the custom dataset. which type of images I need to take for training(for smaller object detection). should I take high-quality images or can I take any kind of images like medium quality and low-quality images? does low-quality images effects the training results?

[Anafeyka]

@praneeth0609 For example, let's say I want to define objects both clear and with a 10/25/50% blur, then my dataset will look something like this.

[praneeth0609]

@Anafeyka thank you for your clarification. for smaller object detection, what should be the object dimension and image size while training. for example, I have a few images attached here.

which type of images should I use for training?

[Anafeyka]

@praneeth0609 It must be something like this Big train Big val

Middle train Middle val

Small Train Small Val

[praneeth0609]

@Anafeyka thank you for your replay, you are supporting me a lot. I have to differentiate UAVs and birds. for example,

the above image has both UAV and bird. how should I differentiate them? as the UAV is bigger than the bird in that image. but consider both bird and UAV are the same sizes in the image, then both look similar. in those cases how should I differentiate them. please suggest me your opinions and suggestions.

[Anafeyka]

@praneeth0609 You should have in the training and validation. As many similar images as possible.

Example Class 0 Drone Class 1 Bird

The more such images you find the more accurate you will get the weights

[Anafeyka]

@praneeth0609 If you look in the upper-left corner of the video 0: 50. You will notice that a small fragment of branches is defined as a face. This is due to a lack of data set images where there would be small branches and small faces. https://www.youtube.com/watch?v=JGm0xM6_p94

[praneeth0609]

@Anafeyka thank you for your suggestions. I want to know some details regarding the training of yolov4. present I'm training custom data using yolov4. this is how I'm getting the loss and mAP graph. can you clarify the bellow questions?

1. How many iterations need to perform(according to the above graph for better results)?
2. what should I do to maintain constant mAP?

[Anafeyka]

@praneeth0609 the best way is to collect 1 training 1 verification data set. And check with the weights already received. Example You have 2 classes of 5000 training images and 700 test images for each class.you train the model for 20,000 iterations. Then you check each point and get the best result. Here it is impossible to answer correctly all depends on the dataset. In any case, my recommendations are not an axiom. You need to experiment.

[Farjad3253]

@praneeth0609 . I will highly recommend you to checkout a YouTube tutorial series of "The AI Guy". He has explained a lot of stuff in an easy way and in one of his tutorial there is an easy way to check the mAP of the trained weights after different iterations. I hope you got the point.

[praneeth0609]

@Anafeyka @Farjad3253 thank you for your suggestions. and I have one more query regarding GPU configuration.

my computer has already installed with GeForce RTX 2060 (6 GB), now I want to increase my systems GPU capabilities. can I add another GeForce RTX 2060 (6 GB)? then it becomes 12GB. if I do this can I train my yolov4 custom data with 608 x 608 with random = 0? Thank you

[praneeth0609]

@Anafeyka @stephanecharette

1. Currently, I'm using a camera for custom object detection using the yolov4 algorithm, which has 1920x1080 pixels resolution. Now I want to change my camera. the specifications of both the new camera and old camera are similar. can I use the same weight files and the same training methods with the new camera?
2. is there any effects or changes in detection with different cameras? (using yolov4 object detection)

[stephanecharette]

1. Yes, if you trained the network with enough variance, it should work fine.
2. Yes, different cameras and CCDs will take different pictures. The focus range may be different, or the crispness of the image, the colour range, lots of things can be different between cameras. But neural networks are normally trained with a fair bit of variance in the pictures, so with any luck you wont need to re-train.

[WilburZjh]

Hi, I am confused on the cfg height and width. Some clarifications would be very helpful. I am running with YOLOv4.

1. My training images are 800 * 800, is that necessary to modify height and width in cfg file to 800 and 800?
2. There are many objects in each image, some of them are big, but some of them are small. After changing the values of anchors by running kmeans, the detection results on big object stays the same, but the small object results did not get a big improvement. Instead of changing the configuration values, any suggestions on changing kmeans to other clustering methods?

[praneeth0609]

@stephanecharette thank you for your clarification.

[praneeth0609]

@stephanecharette I'm using python version of darkent that is darknet_video.py from darknet official repo, with darknet_video.py am able to detect the object with 30fps using yolov4. but for my project i need to add more extensions to darknet_video.py. if I add 2 are more threads for my custom functions to darknet_video.py the frame rate is decreasing to 20 to 25 fps after some time it is decreasing even more. I this this is because of threads. can I use multiprocess concept instead of threads in darknet_video.py. is it make any difference by using process instead of thread.

[shahla-ai]

r tiny-3l configurations, so I couldn't use those instructions. :( See issue #6274.

I did test out YOLOv3-tiny, YOLOv3-tiny-3l, YOLOv4-tiny, and YOLOv4-tiny-3l and then post a video to compare the results of each one: https://www.youtube.com/watch?v=p0Wn8ZNQ_uc

The problem is YOL

hello stephan: can you please refer to which weights file to use for both:

• yolov3-tiny_3l.cfg
• yolov4-tiny_3l.cfg

--

--

[stephanecharette]

can you please refer to which weights file to use for both:

• yolov3-tiny_3l.cfg
• yolov4-tiny_3l.cfg

I train my own networks from scratch. I don't use pre-trained weights.

For example: https://www.ccoderun.ca/programming/2020-03-07_Darknet/

[praneeth0609]

@cenit @AlexeyAB Which is the best preferable GPU for smaller object detection and training with YOLO v4 ..? Currently am using RTX 2070 8 GB for smaller object detection with 608x608. (training) sometimes I faced memory errors. Is this GPU sufficient ...? or should I upgrade to another model for training of smaller object detection. Or which GPU is most suitable for yolov4 training.

Systems configuration : Intel i7 9th gen DDR4 32 GB RAM

Note: training custom data set with yolov4

[stephanecharette]

sometimes I faced memory errors.

Sometimes you have memory errors, and sometimes you don't? That seems a bit unusual.

The 8 GiB RTX 2070 is the same card that I have. There is no GPU that is "best" for smaller object detection. The GPU simply determines how fast it will be, and the amount of memory determines how much can be loaded into memory at once for training. For detection, when you set batch=1 and subdivisions=1 then the amount of memory shouldn't be a problem.

Is this GPU sufficient ...? or should I upgrade to another model for training of smaller object detection. Or which GPU is most suitable for yolov4 training.

No need to upgrade, the RTX 2070 is a great card to use. But make sure you see these:

• https://www.ccoderun.ca/programming/2020-09-25_Darknet_FAQ/#memory_consuption
• https://www.ccoderun.ca/programming/2020-09-25_Darknet_FAQ/#cuda_out_of_memory