Newbie Question

[thealchemist-x]

Hi everyone,

I'm new to deep learning and I want to train my own datasets on top of a pre-trained model here in YOLO. If I may ask 3 questions:

1. What image labeler can I use to specify the ground truth ROI of my dataset?

2. How can I supply the annotated data into this YOLO keras frame work?

3. On which layer in YOLO should I build my dataset on (direction in the code will be helpful)?

Thank you so much!

[Ahziel]

Hi @thealchemist89,

I'm a newbie too but I'm also working on the training of YOLO so maybe I can help you a little with my tiny knowledge.

To create my own dataset with annotations I personally use labelImg but I know that others use BBox-Label-Tool. I prefer labelImg just because you have the annotations in the Pascal VOC format. So, to use your data for YAD2K, you'll have to you a parser to create annotation with the good format. (In python it's easy because the annotation file is in xml)

To train YAD2K you have a script called retain_yolo.py. You just have to provided a .npz file witch contain your data. To create this file, you can used this project witch is provided by @shadySource (who provided also the retrain_yolo.py script).

You just have to execute the script. I don't remember if you need to modified the .cfg for this version, maybe others know that. You also have to created the .txt with your classes.

I had some issues to train my own data because my images had different shapes. If this happen to you too you can check this post to see how solve this problem.

I do not know whether I am clear in my remarks. If you need more explication just ask and if I understand the problem I will answe !

I sorry for my English if it's not clear or if there is a lot of mistakes !

[thealchemist-x]

Thank you Ahziel, you explained it very well. And, I appreciate very much your time and patience in writing so clearly. If I may ask 2 more questions:

1. May I know which post you are referring to that helps us to handle images with different shapes?

2. How can we build our dataset on-top of the pre-trained model of YOLO? I'm trying to do transfer learning hoping not to re-train everything. Do you know where in the code we can do this?

Thank you for helping!

[alecGraves]

@thealchemist89

1. retrain_yolo.py currently resizes (stretches/shrinks) all images and their corresponding labels to a fixed size for easy training with keras. Also, the way training data is loaded now requires all training images to have the same dimensions. The post is near the bottom here.
2. retrain_yolo.py is already configured to load the pre-trained model of yolo then re-train it for another dataset (it already does transfer learning). If you need help understanding how to use it, I could write a small tutorial. The readme does need to be updated with this information.

[thealchemist-x]

@shadySource

Thank you so much for your kind direction. I will follow up based on your suggestions, a tutorial will certainly be great!

[00desire]

I labelled my images using labelImg, just contributing my code to parse the xml files to the format used by @shadySource package_dataset2.py

Hope this can help newbies working with YAD2K :)

import xml.etree.ElementTree as ET
import os

dir = 'data/xmls' #directory containing xml files

f= open("data.csv","w+") #csv file name

for files in os.listdir(dir):
    tree = ET.parse(os.path.join(dir,files))
    root = tree.getroot()

    file_name = root.find('filename').text

    for object in root.findall('object'):
        obj_name = object.find('name').text
        bndbox = object.find('bndbox')
        xmin = bndbox.find('xmin').text
        ymin = bndbox.find('ymin').text
        xmax = bndbox.find('xmax').text
        ymax = bndbox.find('ymax').text
        string = ' '.join([file_name,xmin,ymin,xmax,ymax,'0',obj_name,'\n'])
        f.write(string)

[zq1314]

@00desire hi,can you share your code with me ,here is my emial (1908344230@qq.com)

[MATRIX4284]

First use labelImg-master to convert the boxed pictures into VOC Annotated format the use my utility from the link below to convert the VOC Annotated Files to npz so that you can train your own custom data using YAD2K

https://github.com/MATRIX4284/VOC_NPZ

[priyamehta01]

Hey, I have image text files which I generated from the LabelImg project. I am facing quite some trouble parsing it when there are multiple objects in an image file.

[00desire]

@priyamehta01 refer to the code i provided above using the Element Tree library to parse the xml files to find specific objects.

[Denielll]

@shadySource you mentioned that retrain_yolo.py currently resizes (stretches/shrinks) all images and their corresponding labels to a fixed size for easy training with keras.

However, in the package_dataset.py, u resize img ? Why was that ? I thought retrain_yolo.py already handled it.

What should I do if I have different sizes of img ? Can I label them and then resize ? or I have to resize before labeling ?

Thank you !

[MATRIX4284]

Resizing is a image pre-processing step.It is to bring all image roughly on the same scale so that the matrix gets generic size.You can label and then resize it.Currently retrain_yolo.py handles it .Yoiu can use whatver image size you like it will automatically resize the image into 416,416

Thanks & Regards, Kaustav Technical Envagelist Embedded & Distributed Systems

On Tue, Jul 10, 2018 at 2:27 PM, Denielll notifications@github.com wrote:

@shadySource https://github.com/shadySource you mentioned that retrain_yolo.py currently resizes (stretches/shrinks) all images and their corresponding labels to a fixed size for easy training with keras.

However, in the package_dataset.py, u resize img ? Why was that ? I thought retrain_yolo.py already handled it.

What should I do if I have different sizes of img ? Can I label them and then resize ? or I have to resize before labeling ?

Thank you !

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/allanzelener/YAD2K/issues/54#issuecomment-403751200, or mute the thread https://github.com/notifications/unsubscribe-auth/AT-IAIfafAjMHlKbHiyF1Qm0cZdC9xQkks5uFGyMgaJpZM4ObOSh .

[MATRIX4284]

Hi i saw one thing that the retrain_yolo expects the image to be supplied of the size 416,416.Please resize your images prior to label by PIl or opencv and run the retrain_yolo after proper annotation.

Thanks & Regards, Kaustav Technical Envagelist Embedded & Distributed Systems

On Tue, Jul 17, 2018 at 12:19 AM, Kaustav Mukherjee oracler4284@gmail.com wrote:

Resizing is a image pre-processing step.It is to bring all image roughly on the same scale so that the matrix gets generic size.You can label and then resize it.Currently retrain_yolo.py handles it .Yoiu can use whatver image size you like it will automatically resize the image into 416,416

Thanks & Regards, Kaustav Technical Envagelist Embedded & Distributed Systems

On Tue, Jul 10, 2018 at 2:27 PM, Denielll notifications@github.com wrote:

@shadySource https://github.com/shadySource you mentioned that retrain_yolo.py currently resizes (stretches/shrinks) all images and their corresponding labels to a fixed size for easy training with keras.

However, in the package_dataset.py, u resize img ? Why was that ? I thought retrain_yolo.py already handled it.

What should I do if I have different sizes of img ? Can I label them and then resize ? or I have to resize before labeling ?

Thank you !

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/allanzelener/YAD2K/issues/54#issuecomment-403751200, or mute the thread https://github.com/notifications/unsubscribe-auth/AT-IAIfafAjMHlKbHiyF1Qm0cZdC9xQkks5uFGyMgaJpZM4ObOSh .

[MATRIX4284]

Hi i saw one thing that the retrain_yolo expects the image to be supplied of the size 416,416.Please resize your images prior to label by PIl or opencv and run the retrain_yolo after proper annotation.

Thanks & Regards, Kaustav Technical Envagelist Embedded & Distributed Systems

On Tue, Jul 10, 2018 at 2:27 PM, Denielll notifications@github.com wrote:

@shadySource https://github.com/shadySource you mentioned that retrain_yolo.py currently resizes (stretches/shrinks) all images and their corresponding labels to a fixed size for easy training with keras.

However, in the package_dataset.py, u resize img ? Why was that ? I thought retrain_yolo.py already handled it.

What should I do if I have different sizes of img ? Can I label them and then resize ? or I have to resize before labeling ?

Thank you !

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/allanzelener/YAD2K/issues/54#issuecomment-403751200, or mute the thread https://github.com/notifications/unsubscribe-auth/AT-IAIfafAjMHlKbHiyF1Qm0cZdC9xQkks5uFGyMgaJpZM4ObOSh .

[MATRIX4284]

Hey,

I just signed the petition "Insurance Regulatory and Development Authority of India: Stop rejecting Insurance claim due to hidden clauses after policy is issued" and wanted to see if you could help by adding your name.

Our goal is to reach 1,052 signatures and we need more support. You can read more and sign the petition here:

https://chn.ge/2Jr564L

Thanks! Kaustav

[MATRIX4284]

Hey,

I just signed the petition "Insurance Regulatory and Development Authority of India: Stop rejecting Insurance claim due to hidden clauses after policy is issued" and wanted to see if you could help by adding your name.

Our goal is to reach 1,052 signatures and we need more support. You can read more and sign the petition here:

https://chn.ge/2COjin8

Thanks! Kaustav