How to train own dataset?

[ybdesire]

Which code should I modify if I want to train my own dataset?

• different classification category
• different image size

[ybdesire]

I want to train my own dataset

• Which config should we modify ?
• Should we run kmeans ?

[fnando1995]

Hello @ybdesire i also want to train yolov3 for my own dataset. But have some difference. I want to ADD a class... any help with this?

[ybdesire]

Hi @fnando1995

I still not get the detailed steps for training own dataset. But for your case, I think you can try:

1. add your class name into https://github.com/qqwweee/keras-yolo3/blob/master/model_data/voc_classes.txt
2. your class_id is 21, so you can label your image as

   path/to/img2.jpg 120,300,250,600,21

3. follow the training steps at here https://github.com/qqwweee/keras-yolo3#training

[Fingerl]

@ybdesire ，hello，in my view，if you want to train the classes doesn't in the coco,you don't need run the convert.py,in the other word,once you get your own weight.h5,you can load it directly in you next training instead of always run the convert.py . Of course，you need modify the classes.txt.Do you think?

[fnando1995]

YES! Thanks for your answers.

I already did that and everything works fine. I use Annotation Tools from @ManivannanMurugavel to generate the train.txt archive.

Then i change some dirs and make it traing at 16 batch size.

It detects and tracks as expected.

[deepmeng]

@fnando1995 hello,man. I'm curious whether the original weight is effective for your training when you change the numbers of output category ？ If not, how you solve it?

[fnando1995]

@fnando1995 hello,man. I'm curious whether the original weight is effective for your training when you change the numbers of output category ？ If not, how you solve it?

I dont understand your questions.

If you ask for how i did the training, its pretty simple.

You have a train.py file that does all the work. But you frist have to give it data (boxes and images). The strange for me was the boxes cause i didnt understand this at first.

The boxes of all your images are refered in one file. This file have the following format:

PATH/TO/IMAGE/1.jpg x1,y1,x2,y2,class x1,y1,x2,y2,class etc...

This means that every line is one image and EVERY box in the image as (X1,Y1)(X2,Y2) and the class. NOTE that if you are using the weight of cocodataset, you need the classes.txt file modify, adding the new class at the bottom. 1 person.. 2... 80 (LAST COCO CLASS) 81 YOUR NEW CLASS 82 YOUR SECOND NEW CLASS ...

REMEMBER YOU START WITH 0!!!

Hope i help u

[deepmeng]

Thanks for your careful explaining！ The "original weight" means the yolov3.weights. When you change the class number, do you also change the yolov3.cfg and convert the yolov3.weights to *.h5 based on the changed yolov3.cfg?

[fnando1995]

NO! When you use the convert.py you are just trying to convert one type of file to other type. In this case the .weight + .cfg to .h5. i suggest that AFTER you have the .h5 start the retraining for any other class you want.

[deepmeng]

Many thanks!

[fabero]

@fnando1995 Could you explain why there are suddenly numbers assigned to each class? The coco_classes.txt does not seem to contain any numbers. Or do you simply mean the line numbers? If so, what do you mean with 'Remember you start with 0'?

[fnando1995]

As you say, simply mean the line numbers, but! inside the code, the class is worked, used or referenced as the numbered row in the file. that means that the first class (person) is referenced as cero (0). With that in mind, if you have 80 classes written in then file, those classes are referenced from 0 to 79. if you add a new class then is referenced as 80.

Hope being helpfull.

[fabero]

Thankyou for your explanation!

Do you also know if it is possible to remove classes from that list? If so, would that maybe improve things like training time and/or accuracy on the other classes?

[fnando1995]

Thankyou for your explanation!

Do you also know if it is possible to remove classes from that list? If so, would that maybe improve things like training time and/or accuracy on the other classes?

I am aware it could be done. But i haven't done it.

As far as i have read, i dont think it actually improve training time. Because that trainning have being done with other data, and over that training you are re-training. neighter accuracy cause it actually, for me it depends of the data you give to both the training and the validation.

If you retraining with new data, it will not improve accuracy in the previous trained classes neighter.

Hope being helpfull

[bodhwani]

Hi @fnando1995. Can you please mention that in which file do we need to add image name and boxes coordinates? Also from where do we get yolo_weights.h5?

[fnando1995]

@bodhwani Hello!

Ok, go to the 'train.py' file

""" annotation_path = 'train_centos7.txt' log_dir = 'logs/005/' classes_path = 'model_data/classes/coco_classes.txt' anchors_path = 'model_data/anchors/yolo_anchors.txt' """ This file have the annotation_path var, This file actually have this: """ /home/user/..../1.jpg 151,365,256,401,80 /home/user/..../2.jpg 30,120,69,952,80 40,10,169,152,80 /home/user/..../3.jpg 90,326,326,156,80 . . . """ note: '...' means the path, not actually a directory....

Check that this file have an image path in every line followed by space, 5 number, space, 5number and so on... this means that for image2.jpg there is one objectat 30,120,69,952 of class 80, and other object at 40,10,169,152 of class 80.

When training, the file is cheacked and each line trains the model.

yolo_weights.h5 is a keras file, you can download and convert the yolo.cfg and yolo.weights from the oficial website of yolo and convert it to yolo.h5 (extension of keras) with the convert.py file of this github.

hope this helps.

[bodhwani]

Thanx @fnando1995 I have followed these two steps:

wget https://pjreddie.com/media/files/yolov3.weights python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5

After converting, while running train.py, I got an error: ` keras-yolo3/yolo3/model.py", line 376, in yolo_loss

anchors[anchor_mask[l]], num_classes, input_shape, calc_loss=True) IndexError: index 5 is out of bounds for axis 0 with size 5`

My train.txt file is like this: /Users/images/147772025319520161028_151915.png 22,613,150,354,0 /Users/images/147742629330620161025_155320.png 125,499,20,268,0 /Users/images/JPEG_20160527_155124_1000980290136.png 154,467,0,480,0 /Users/images/JPEG_20161028_121752_10003102030.png 134,546,161,349,0 /Users/images/1474638701993DSC06811.png 161,573,43,382,0

(total 5 entries for now) coco_classes.txt: person`

yolo_anchors.txt: 0.57273, 0.677385, 1.87446, 2.06253, 3.33843, 5.47434, 7.88282, 3.52778, 9.77052, 9.16828

Can you please help? Thanx

[fourth-archive]

@ybdesire @fnando1995 @Fingerl @deepmeng @fabero this YOLOv3 tutorial may help you: https://github.com/ultralytics/yolov3/wiki/Train-Custom-Data

The accompanying repository works on MacOS, Windows and Linux, includes multigpu and multithreading, performs inference on images, videos, webcams, and an iOS app. It also tests to slightly higher mAPs than darknet, including on the latest YOLOv3-SPP.weights (60.7 COCO mAP), and offers the ability to train custom datasets from scratch to darknet performance, all using PyTorch :) https://github.com/ultralytics/yolov3

[ybdesire]

@fourth-archive Thanks for the doc. But I'd like to train yolov3 the keras version. Do you have any suggestion ?

[abhay8051]

@fnando1995

Can you help me with the bounding box format

The required format is x_min,y_min,x_max,y_max,class_id - the box coordinates are 2 corners of the box but @ManivannanMurugavel annotation tool gives bounding boxes in this format x y width height. which is the center and the length of 2 sides

also the numbers should not be normalized for image size as that is handled in train.py can you throw some light on the required format ,

is there any alternate tool for annotation

[fnando1995]

@ybdesire I think you can convert it. I have lost some track of this git. Sorry for not better answer.

@abhay8051, hello man. You can transform from x y width height to x_min,y_min,x_max,y_max. Use some maths. You got the centroid and the witdh and height.

[ManivannanMurugavel]

Hi @abhay8051, Please use my new yolo annotation tool. Because it will return the yolo format class_id,x_min,y_min,x_max,y_max not x,y,width and height. I have mentioned the new annotation tool below. You can use it. Link: https://medium.com/@manivannan_data/yolo-annotation-tool-new-18c7847a2186

[abhay8051]

@fnando1995 and @ManivannanMurugavel Thanks lot guys for help (here and everywhere on the www) to figure this out. I took a bit loong way but got it working. This is what I did

1) i found labelimg v1.8.1 very helpful as i had multiple classes in same training image 1a) I didnt planned in advance, so started annotating images with Yolo option. This option in the tool provides a [class id x y w h] format. it creates 1 txt file for each image 2) this implementations needs [xmin,ymin,xmax,ymax,class id] , 1 txt file for all images and annotations 2a) So i used a yolo to voc converter. this would convert the .txt files from 1a above to .xml files with required encoding. Converter here 3) finally used the voc_annotation.py with a little tweaking to create that single text file with all annotations.

it took me 4 days to figure this all out, hence sharing it here :) Further, I customer trained the yolov3 algorithm today for 6 new classes (i did not added/extended the existing classes). It works good, but iam yet to measure the KPI's

[cowboycyx]

Hi Guys,

I am also trying to train my own model to detect pen. I really appreciate if you could help me with the following questions:

1 How many images should be enough for adding one new class? In my case, how many pens should I labelled before conducting the training? 2 The yolo.h5 is trained with coco dataset, so do I need to download coco dataset, and train the model with coco image dataset along with my labelled pen images? 3 Normally, the box of pen is a slim rectangle if the pen is located vertically or horizontally in an image. However, if the pen is located with an angle, saying 45 degree towards the image edge, my labelling tool can only make a square-like box to label it. Does this kind of labelling have a bad effect on the final detection results? If so, does it mean I should only use those images with a vertical or horizontal pen?

Thanks very much.

[fnando1995]

@cowboycyx Hello,

1) There is no exactly number for that. too many pictures will give a bias to the net, and too few will make the net not to detected precisely. You must test. But i could recomend if you are only using it to detect one single object, at least 400 labeled pictures.

2)The yolo.h5 is already trained in coco dataset, so it has the weights of this trainning. What you are about to do is fine tunning for your own output(s) (pen and/or others maybe). Using the weights of the coco dataset will help you in this training cause the loss will reduce faster. To resume: no you do not need to train the net with you dataset and coco dataset, only yours.

3) No, actually is ok, the detection is just trying to "detect" it inside the image, if you try to perform a pixel by pixel detection of he image, i recomend semantic segmentation. dont worry about the extra data in the boxes, it will not complicate much the trainning.

Hope this help you.

[cowboycyx]

@fnando1995

thanks for you kind and detailed reply, it help a lot.

Regards.

[fnando1995]

@cowboycyz. Your welcome

[abhay8051]

@cowboycyx

3 if you do not change the default anchor boxes, you might have issues with detection. Eg. a vertical pen will need a vertical Bounding Box whereas a diagonal or horizontal pen will need a bigger horizontal box. The person related anchor boxes might work but they will be a bit broad (fat) for a pen. Try the default ones first, before experimenting.

[hasansalimkanmaz]

even though I changed my classes txt for my classes. Trained model keeps going on detecting classes from COCO dataset. Can we train a model that only detects specified classes from this repo?

[zhoujinhai]

@fnando1995 hello, I want to ask your a question, my train.txt file like this, but output the box_data in utils.py are zeros list.

VOC2012/JPEGImages/000000.jpg  49,177,6,144,0  113,283,175,328,0  312,492,95,240,1 
VOC2012/JPEGImages/000001.jpg  196,327,52,187,0  259,397,225,370,0  327,416,136,221,1  281,322,13,115,1 
VOC2012/JPEGImages/000002.jpg  116,221,83,201,0  304,369,27,85,1  155,245,17,71,1

box:  [[176 248  69 151   0]
 [400 450 149 229   1]] len box:  2
box_data:  [[0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0.]] len:  20

[bd7jat]

@fnando1995 hello,man. I'm curious whether the original weight is effective for your training when you change the numbers of output category ？ If not, how you solve it?

I dont understand your questions.

If you ask for how i did the training, its pretty simple.

You have a train.py file that does all the work. But you frist have to give it data (boxes and images). The strange for me was the boxes cause i didnt understand this at first.

The boxes of all your images are refered in one file. This file have the following format:

PATH/TO/IMAGE/1.jpg x1,y1,x2,y2,class x1,y1,x2,y2,class etc...

This means that every line is one image and EVERY box in the image as (X1,Y1)(X2,Y2) and the class. NOTE that if you are using the weight of cocodataset, you need the classes.txt file modify, adding the new class at the bottom. 1 person.. 2... 80 (LAST COCO CLASS) 81 YOUR NEW CLASS 82 YOUR SECOND NEW CLASS ...

REMEMBER YOU START WITH 0!!!

Hope i help u

hello,fnando1995, i also use yolo-annotation-tool to creat labels ,but i found i can not gennerate txt files as below: /home/user/..../1.jpg 151,365,256,401,80 /home/user/..../2.jpg 30,120,69,952,80 40,10,169,152,80 /home/user/..../3.jpg 90,326,326,156,80 (i believe this formart is yolov3 need for training) But what i got form label folder are txt files ,each of them has only one line as below 0 0.50862 0.54678 0.72 0.8012532 Did i do something wrong with this tool ? or i need some extra tool to finsh the rest job? Thanks

[masterhimanshupoddar]

Here is a step by step to train on your own dataset. Lets say you have multiple images in your images directory. Lets start from the beginning.

1. Download https://github.com/qqwweee/keras-yolo3

2. I suppose you know how to annotate your images using tools(labelImg) and then you have created the text file for all of the images, with images in them in the following format.

   /home/user/..../1.jpg 151,365,256,401,80
   /home/user/..../2.jpg 30,120,69,952,80 40,10,169,152,80
   /home/user/..../3.jpg 90,326,326,156,80

   now rename this file as train.txt and place it in the keras-yolo3 folder.

3. Download yolov3.weights from their official website, move it to the keras-yolo3 folder.

4. Convert this pre-trained model to h5 by python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5

5. Now before training we need to create new anchors for our custom dataset. open kmeans and add this line wherever you find filename =, filename = "train.txt" if your train.txt is in the keras-yolo3 folder just run this, and it will create it for you. python kmeans.py

6. Move the yolo_anchors file created to the model_data folder.

7. Now add the classes to the coco_classes.txt file, check your train.txt file if the classes present in your image starts from 0,1,2... in the train.txt file then add your classes in the beginning as

   A
   B
   C
   ...
   ...
   person
   bicycle
   car
   motorbike

8. Now we are ready to go, lets train our model.

9. Open train.py and verify these things

   
   annotation_path = 'train.txt'
   log_dir = 'logs/000/'
   classes_path = 'model_data/coco_classes.txt'
   anchors_path = 'model_data/yolo_anchors.txt'
   weights_path='model_data/yolo.h5'

weights_path is the path of your h5(step 4) file
Train as `python train.py`

[fnando1995]

@masterhimanshupoddar Hello, really clean your explanation.

I have just one question. I hadnt change he anchor for retrainning any time, and it had gone well. I don't know what are the anchors for. Are they for normalize a size for the objects inside the image? I think that with this maybe a full-screen bounding box will be imposible? Am I correct or is there any other reason for changing the anchors?

[masterhimanshupoddar]

I have just one question. I hadnt change he anchor for retrainning any time, and it had gone well. I don't know what are the anchors for. Are they for normalize a size for the objects inside the image? I think that with this maybe a full-screen bounding box will be imposible? Am I correct or is there any other reason for changing the anchors?

@fnando1995 Though its optional to change the anchors but since your images dimensions may be different from what YOLO was originally trained on so its better to give your own anchors for your dataset.

Also can you help me out with this one https://github.com/qqwweee/keras-yolo3/issues/635

[fnando1995]

I have just one question. I hadnt change he anchor for retrainning any time, and it had gone well. I don't know what are the anchors for. Are they for normalize a size for the objects inside the image? I think that with this maybe a full-screen bounding box will be imposible? Am I correct or is there any other reason for changing the anchors?

@fnando1995 Though its optional to change the anchors but since your images dimensions may be different from what YOLO was originally trained on so its better to give your own anchors for your dataset.

Also can you help me out with this one #635

Got it!, I will check you thread.

[aimxu]

@masterhimanshupoddar Thx for this Concise and effective reply I have some questions about step 7 to ask

1. According to it, if I have 3 classes (traffic sign) to train, is the new 'class.name'(the context is all classes name) contain 83 lines because of the weights base on coco dataset?

2. The order of new classes name should be at the top of 'class.name'? Can it be on the bottom of 'class.name'? or shffling in the originnal file? Why?

3. My classes have nothing to do with coco, can it trained on the pretrained weight?

4. Can train it without the pretrained weight? How

And the last question is nothing to do with step 7, I can't see the dataset folder parameter, how can it get the data? through the file path in Annotation file?

Thanks again

[masterhimanshupoddar]

@aimxu

1. Yes since you have 3 extra classes, so it will contain 83classes
2. Check your train.txt file if the classes are alotted numbers as 0, 1, 2 then it should be added to beginning.
3. Yes
4. Yes, but I won't suggest you to, search on google how to do it

[aimxu]

@masterhimanshupoddar Thanks a lot especially about the answer of no 2 question Now I have trained some traffic sign data, but the loss is a little hign after 10 epoch, about 7, and it remains 7-8 since epoch 4, maybe my training method is not so appropriate, I will train without pretrain weight on through google, thanks a gain.

[fnando1995]

@aimxu Hello! I also tried to train 3 classes before. I recommend:

Change class.name completely. like:

sign1
sign2
sign3

No matter if you use the weigths of previous traininng. This is learning transfer. You use the weigths of a previous trainning to train new data, but the last layers of the network will change (the ones that give the outputs) this will have 3 instead of 83. This is better because for the new trainning you will not use the other 80 classes in coco dataset (I assume), BUT! the "magic" done by the previous training will stay at the frist layers (where weights,bias are already "centered" to the purpose of the network).

This is actually more complex than what i explain. keep that in mind.

[aimxu]

@fnando1995 Indeed When we apply the convert command, python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5 It means that the yolo.h5 contains 80 classes, And when we apply python train.py with 3 classes, the new weight's classes change to 3 automatically without changing 'yolov3.cfg', it change with code. It is right? And I will train a again with 3 classes, it should be better! thank you very much!

[fnando1995]

No, you must change .cfg filebecause thats the architecture of the PRESENT training you are goign to do.

Remember that this files is distributed to know how it was built. If you pretend to change the outputs then you should change this file too (.cfg file). This does not mean that you will change the weights/bias, thats inside the .weights file which is (Let''s say) the ACTUAL state of learning/knowledge of the network. If you use it again (.weigths file) to retrain the network (transfer learning), you will just ajust this learning/knowledge for a new TASK (detecting the new objects).

[aimxu]

command python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5 is the only place where using .cfg file (so far as I found, I have found nowhere else to use .cfg file including codes ). So must I change the .cfg file before applying the conver.py command? @fnando1995

[aimxu]

@fnando1995 And the anchor in .cfg file need to change to the anchor created by kmeans.py? Or remain the original one?

[fnando1995]

@fnando1995 And the anchor in .cfg file need to change to the anchor created by kmeans.py? Or remain the original one?

I trained it with out changing anchors. In fact, there isn't a relation between cfg and anchor written in those files. As far as I remember, you initiate each in different lines in train.py.

[aimxu]

@fnando1995 Yes, in tain.py there is a parameter call anchors_path also in yolo.py I have change anchor in .cfgtoo, and have train 10 epochs to test I will traing without changing anchors in .cfg and compare to today's training. Long way to go to comphense YOLO completely, Thanks very much!

[fnando1995]

@aimxu you are rigth. I havent notice that inside .cfg file were also the anchors. Even though this, I trained this repo with my dataset without changing the anchors (just using the correct one; anchor to yolo / anchors-tiny to yolo-tiny, not especifically the real names of the files).

Give it a try.

[SrikarNamburu]

Here is a step by step to train on your own dataset. Lets say you have multiple images in your images directory. Lets start from the beginning.

1. Download https://github.com/qqwweee/keras-yolo3
2. I suppose you know how to annotate your images using tools(labelImg) and then you have created the text file for all of the images, with images in them in the following format.

/home/user/..../1.jpg 151,365,256,401,80
/home/user/..../2.jpg 30,120,69,952,80 40,10,169,152,80
/home/user/..../3.jpg 90,326,326,156,80

now rename this file as train.txt and place it in the keras-yolo3 folder.

1. Download yolov3.weights from their official website, move it to the keras-yolo3 folder.
2. Convert this pre-trained model to h5 by python convert.py yolov3.cfg yolov3.weights model_data/yolo.h5
3. Now before training we need to create new anchors for our custom dataset. open kmeans and add this line wherever you find filename =, filename = "train.txt" if your train.txt is in the keras-yolo3 folder just run this, and it will create it for you. python kmeans.py
4. Move the yolo_anchors file created to the model_data folder.
5. Now add the classes to the coco_classes.txt file, check your train.txt file if the classes present in your image starts from 0,1,2... in the train.txt file then add your classes in the beginning as

A
B
C
...
...
person
bicycle
car
motorbike

1. Now we are ready to go, lets train our model.
2. Open train.py and verify these things

    annotation_path = 'train.txt'
    log_dir = 'logs/000/'
    classes_path = 'model_data/coco_classes.txt'
    anchors_path = 'model_data/yolo_anchors.txt'
    weights_path='model_data/yolo.h5'

weights_path is the path of your h5(step 4) file Train as python train.py

My custom dataset has 15 classes, so before running convert.py should I edit the yolo.cfg or not?

[fnando1995]

@SrikarNamburu no needed. Convert.py change from darknet to keras your model.

[SrikarNamburu]

@SrikarNamburu no needed. Convert.py change from darknet to keras your model.

The architecture in yolov3.cfg is defined for 80 classes, right? Also, I have edited 'model_data/coco_classes.txt' to my dataset -- removed 80 classes and have put only my 15 classes.

[robisen1]

@SrikarNamburu no needed. Convert.py change from darknet to keras your model.

My data set is from a drone. This means the images are from above. Will I need to also unfreeze all the layers?