flamma7
commented
5 years ago We haven't trained a new YOLO model in a few months b/c it's not easy to. So the whole idea behind YTS is that bag files -> Labeled images in a zip -> new model file very quickly (pretty much by running 3-4 scripts). Here's the order of events:
- decompress & turn bag into jpgs
- Split jpgs into zips with 500 images each to allow for ppl labeling in parallel
- Label jpgs using OpenLabeling included in YTS
- Recombine distributed labeled zips into a master labeled zip
- Split positives & negatives. Maximum number of images in a zip is 500. We do this for example so that when you go to train a model and you download a labeled "dice" zip file. You don't get 10 labeled and 490 unlabeled images, you actually get 500 positives, and you'll go to a separate place to get 500 negatives. This helps the user get a better idea of how much data they're including in training their model. Note that YTS will inform the user of how many pos and neg examples it has before training
- Name pos & neg zip files according to YTS nomenclature, upload to the google drive in the correct spots Besides the 3rd bullet point it's inefficient for us to do any of this by hand.
- Now leaving the data_gen directory and going to the model_gen directory. The other things I've mentioned will save some time, this next is where YTS really shines in its effectiveness: Each jpg has a corresponding txt file with the class number and location of a bounding box on each new line, see the following example T0_9-1-18-image0_0060.txt The 0 in the first column is the class number. Note that OpenLabeling assigns this class number at run time based on the order you specify the classes you are labeling. What this essentially means is that if I'm training on a set of dice6 and roulette table, dice6 may get the class number 0. But if I'm training on a set of images with start gate and dice6, open labeling might assign dice6 a class number of 1. If darkent's understands startgate as 0 and dice6 as 1 that means in the first dataset it'll think all the dice6 images are start gates! @derekwright29 is writing a script that automatically will resolve this conflict and transform all of the data referring to the same object to the correct class darknet expects before feeding it into training darknet. I have no idea how this was done previously, maybe because we previously were only training on a few datasets this wasn't the royal pain it will become soon if we don't automate it This final script pulls from a class assignment file located in the top directory of YTS essentially saying that class 0 will always be start gate, dice 5: 1, dice6: 2 ....
Does this just mean we have a bunch of people classify all our data? Or does this mean we make a system for letting multiple people classify data?
I think we may want to be more robust with our classification. Right now we are just using Darknet but if we want to move to a more complex segmenting neural network we want more accurate outlines than just bounding boxes. Bounding boxes can be derived from a polygon outline so it might be beneficial to create our own labeler that does that and a script for converting to Darknet so we can rapidly train a more advanced neural network. This would be very useful in orienting ourselves in a fine object manipulation task and might reduce the tuning that comes with extracting that data with classical CV.