Retraining Pretrained Inception v3 model

[ghost]

Hi @gkrasin!

Is there a way to retrain (fine-tune) the pretrained classifier https://github.com/openimages/dataset/wiki/Running-a-pretrained-classifier with other datasets? For example, we want to use this one https://www.microsoft.com/en-us/research/project/msra-cfw-data-set-of-celebrity-faces-on-the-web/.

If yes, what shall I do? What structure should a new dataset have? What commands should I execute to train the model with new images?

[gkrasin]

@okasanasan sure! The checkpoint is fully compatible with retraining / finetuning. Give it a try.

[gkrasin]

As for the exact instructions, I would refer to TensorFlow for Poets tutorial. You will need to change the retrain.py script to load the OpenImages checkpoint, but the general process is the same.

[ghost]

Thanks, @gkrasin. Can you help me how can I give it a try? Shall I use this approach https://github.com/tensorflow/models/tree/master/inception?

[gkrasin]

@okasanasan yeah, that should work (after a few changes in the scripts).

[ghost]

As for changes, I think, I need to point somehow on dict.csv and labelsmap.txt, so these files will be modified with new words (categories). Am I right?

[gkrasin]

@okasanasan sorry, I have not looked into this close enough to say anything with confidence. If you make it working, please, report your experience back, so that others can follow your way.

[ghost]

I've tried to retrain it with Inception, but it didn't work. I didn't change anything in the files, but I don't believe I'll be able to do that by myself.

And as for https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/image_retraining/retrain.py. This script requires different format files: classify_image_graph_def.pb, imagenet_synset_to_human_label_map.txt, and imagenet_2012_challenge_label_map_proto.pbtxt.

And this classifier has only model.ckpt file.

So I need to covert .ckpt into .pb, .txt and .pbtxt, right?

[gkrasin]

I didn't change anything in the files, but I don't believe I'll be able to do that by myself.

Some changes will be required. While I (as I have stated earlier in the thread) have not looked at this close enough, and can't tell what exactly needs to be changed, I can try to talk to the people behind retrain.py script and find a proper way to integrate the Open Images baseline model with it.

I could not promise any specific dates or outcome, but I promise to work on it until it clicks.

[ghost]

@gkrasin, thank you!!!

[dineshbvadhia]

I've checked the site but don't understand if the pre-trained InceptionV3 model is just of the image annotations (ie. text) or of the actual images or both? Thx.

[gkrasin]

@dineshbvadhia the pretrained model is a TensorFlow graph with its weights stored in a checkpoint file. It does not include nor text, nor images, but provides a way to take an image as a input and compute predictions on it.

[dineshbvadhia]

@gkrasin Maybe I'm confused by the terminology. How is this - " a TensorFlow graph with its weights stored in a checkpoint file" - different from a TensorFlow InceptionV3 ImageNet pre-trained model?

[gkrasin]

@dineshbvadhia three primary differences:

1. This model has different weights values as it was trained on a different set of images (OpenImages vs ImageNet)
2. Different (and much larger) set of labels
3. It's a multi-label network, as opposed to a single label ImageNet.

Otherwise, it's very similar.

[dineshbvadhia]

@gkrasin Ok, got it. I use keras (with a TensorFlow backend) to extract abitrary layers from the ImageNet InceptionV3 pre-trained model. Is that possible with the OpenImages model?

[gkrasin]

Most likely possible. After all, it's a regular Inception v3 with a slightly modified output layer (sigmoid vs softmax). I don't have any specific advice, though.

[dineshbvadhia]

Is there a Google Groups for OpenImages, if not, is one planned to ask questions outside of Github Issues?

[gkrasin]

@dineshbvadhia no immediate plans to create a mailing group. May be at some point in the future (after the next release)

[dineshbvadhia]

A Google Group would be welcome. Is the roadmap for OpenImages public ie. what is coming down the road?

[gkrasin]

Is the roadmap for OpenImages public ie. what is coming down the road?

Not yet. Hopefully, it will be at some point.

[NerminSalem]

i m trying to work with the retrain.py file but i have a problem with tensorboard i worked but with nothing to display, i have tried the solution of tensorflow/readme file but didn't help so can anyone helps me? thanks

[NerminSalem]

forget to mention that i m following this tutorial https://www.tensorflow.org/tutorials/image_retraining

[trumvu]

What is the accuracy (top-1 and top-5) of this pre-trained Inception-v3 model?

[gkrasin]

@trumvu it needs to be noted that since the dataset is multi-label (an image may have and usually has several annotations). That's unlike ImageNet where each image has a single class. That makes it hard to use the same metrics (like, top-1 and top-5). Their exact definitions do not make sense anymore.

Instead, the net might be considered as the large collection of binary classifiers, one for each entity type (and it's >6K of them). Then each binary classifier has its own accuracy and recall. Some numbers are available on the main page: https://github.com/openimages/dataset#stats-and-data-quality

More numbers are available in this paper: Learning From Noisy Large-Scale Datasets With Minimal Supervision

[kinhunt]

"Some changes will be required." to retrain the V3 model. Can you share some more specific details or even code? It would be very helpful. Thank you anyway.