How to run your code to mark one's name in one image?(accomplish face recognition)

[Victoria2333]

If i want to test one image,and the result will cirle out the faces in the image and label his or her name beside the bounding boxes,so how to run this code to train my own model and how to modify this code to accomplish my goal? Thanks,very much!

[MaartenBloemen]

The explanation on how to train a classifier on your own images can be found on the wiki. As for the first question, do you want to draw the bounding box and name on the face in the picture? If so that can be easily done with OpenCV.

[Victoria2333]

i mean could the existing code draw bounding box and name on the face?coz...the experiment result are just some assessment index.

[MaartenBloemen]

I'm afraid that's not implemented, you will need to modify the code from classifier.py to suit your purpose.

[Victoria2333]

Thanks for ya~~~

[MaartenBloemen]

Do you know how to do this? Otherwise I'm willing to help you with this.

[Victoria2333]

Oh my god ,actually,i am not capable of this,it is so nice of you!!! Now,i want to load the pretrained model and look up to its network structure,but i don't know how to show its network structure and parameters' weights. After this,i want to modify the final layer and softmax layer to train my own face datasets. Do you have some suggestions or are you willing to help me? Sorry to reply you lately and thanks very much ~~~

[Victoria2333]

@MaartenBloemen
Or in other words,if i want to modify final layer and softmax layer of his network,should i reload pretrained model and add some command to modify it or does the david use inception_resnet_v1.py to train 20170512-110547(pretrained model) and maybe i just need to make some changes in this .py file???

[MaartenBloemen]

I will just post the script here as I'm to lazy to create a repo for it. The code it's the cleanest or the best, but it works:

import tensorflow as tf
import numpy as np
import argparse
import facenet
import math
from scipy import misc
import align.detect_face
import pickle
import cv2

def classify(args, pnet, rnet, onet):
    _pnet = pnet
    _rnet = rnet
    _onet = onet
    BATCH_SIZE = 90
    IMAGE_SIZE = 160
    MARGIN = 44

    with tf.gfile.FastGFile(args.model, 'rb') as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        _ = tf.import_graph_def(graph_def, name='')

    with tf.Session() as sess:
        images_placeholder = sess.graph.get_tensor_by_name("input:0")
        embeddings = sess.graph.get_tensor_by_name("embeddings:0")
        phase_train_placeholder = sess.graph.get_tensor_by_name("phase_train:0")
        embedding_size = embeddings.get_shape()[1]

        print('Calculating features for images')
        img = cv2.imread(args.image)
        images = [cv2.cvtColor(img, cv2.COLOR_BGR2RGB)]
        nrof_images = len(images)
        nrof_batches_per_epoch = int(math.ceil(1.0 * nrof_images / BATCH_SIZE))
        emb_array = np.zeros((nrof_images, embedding_size))
        for i in range(nrof_batches_per_epoch):
            start_index = i * BATCH_SIZE
            end_index = min((i + 1) * BATCH_SIZE, nrof_images)
            images, boxes = align_data(images, IMAGE_SIZE, MARGIN, _pnet, _rnet, _onet)
            feed_dict = {images_placeholder: images, phase_train_placeholder: False}
            emb_array[start_index:end_index, :] = sess.run(embeddings, feed_dict=feed_dict)

        classifier_filename_exp = args.classifier

        print('Testing classifier')
        with open(classifier_filename_exp, 'rb') as infile:
            (model, class_names) = pickle.load(infile)

        print('Loaded classifier model from file "%s"' % classifier_filename_exp)

        predictions = model.predict_proba(emb_array)
        best_class_indices = np.argmax(predictions, axis=1)
        best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices]

        for i in range(len(best_class_indices)):
            print('%4d  %s: %.3f' % (i, class_names[best_class_indices[i]], best_class_probabilities[i]))
            img = cv2.rectangle(img, (int(boxes[i][0]), int(boxes[i][1])),
                                  (int(boxes[i][2]), int(boxes[i][3])), (0, 255, 0),
                                  2)
            img = cv2.putText(img, (class_names[best_class_indices[i]] + " - " + str(
                "{:.2f}%".format(best_class_probabilities[i] * 100))),
                                (int(boxes[i][0]), int(boxes[i][1] - 10)),
                                cv2.FONT_HERSHEY_SIMPLEX, 0.5,
                                (0, 255, 0), 1, cv2.LINE_AA)
            cv2.imshow('Picture classifier', img)
            res = cv2.waitKey(0)
            while res != 27:
                res = cv2.waitKey(0)

            exit(0)

def align_data(image_list, image_size, margin, pnet, rnet, onet):
    minsize = 20  # minimum size of face
    threshold = [0.6, 0.7, 0.7]  # three steps's threshold
    factor = 0.709  # scale factor

    img_list = []
    boxes = []

    for x in xrange(len(image_list)):
        img_size = np.asarray(image_list[x].shape)[0:2]
        bounding_boxes, _ = align.detect_face.detect_face(image_list[x], minsize, pnet, rnet, onet, threshold, factor)
        nrof_samples = len(bounding_boxes)
        if nrof_samples > 0:
            for i in xrange(nrof_samples):
                if bounding_boxes[i][4] > 0.95:
                    det = np.squeeze(bounding_boxes[i, 0:4])
                    bb = np.zeros(4, dtype=np.int32)
                    bb[0] = np.maximum(det[0] - margin / 2, 0)
                    bb[1] = np.maximum(det[1] - margin / 2, 0)
                    bb[2] = np.minimum(det[2] + margin / 2, img_size[1])
                    bb[3] = np.minimum(det[3] + margin / 2, img_size[0])
                    cropped = image_list[x][bb[1]:bb[3], bb[0]:bb[2], :]
                    aligned = misc.imresize(cropped, (image_size, image_size), interp='bilinear')
                    prewhitened = facenet.prewhiten(aligned)
                    img_list.append(prewhitened)
                    boxes.append(bounding_boxes[i])

    if len(img_list) > 0:
        images = np.stack(img_list)
        return images, boxes

    return None, None

def create_network_face_detection():
    with tf.Graph().as_default():
        gpu_options = tf.GPUOptions(allow_growth=True)
        sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options, log_device_placement=False))
        with sess.as_default():
            pnet, rnet, onet = align.detect_face.create_mtcnn(sess, None)
    return pnet, rnet, onet

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument('model', type=str,
                        help='Path to a model protobuf (.pb) file')
    parser.add_argument('classifier',
                        help='Path to the classifier model file name as a pickle (.pkl) file.')
    parser.add_argument('image',
                        help='Path to the image file to classify.')

    args = parser.parse_args()

    pnet, rnet, onet = create_network_face_detection()

    classify(args, pnet, rnet, onet)

You can copy paste this code into a python file called classify_image.py and then run it using this command: python classify_image.py /path/to/model.pb /path/to/classifier.pkl /path/to/image/to/classify.jpg

That being that I don't really see why you would want to modify the last layer, in my first post I mentioned a page on the wiki, check that out it explains how to train a classifier on your own dataset, than you can use that pkl file with this script.

Hope this helps you!

[Victoria2333]

@MaartenBloemen,Thank you very much!!! And as for modifying the last layer,coz i want to train about 100 people ,and i don't know what's the network layer like in this pretained model,maybe the number of neuro cells of last layer(in other words the class numbers) isn't compatible with my face project... And since you mention your page on the wiki,i will look through it carefully~~~ Another little request,could you give me your gmail address or something else to keep in touch??I am very grateful for your help and maybe ask you some academic questions if you are convenient~~~

[Victoria2333]

@MaartenBloemen,I have looked through the page on the wiki,it seems there is nothing to do with the class numbers,so i don't have to modify the model??? I don't know if i get the right understanding......

[MaartenBloemen]

Yes if you follow the guide on the wiki you don't need to retrain the final layer of the model. You can use the pickle (.pkl) file you get from the guide on the wiki with the script I wrote to predict people in an image. You can send me emails on bloemen.maarten@hotmail.com, though I'm just college student and by far no machine/deeplearning expert, but if you send me a mail on that address I'll try my best to help you with your questions.

[ha1990-12]

@MaartenBloemen Thank you for sharing this. But it seems that there is no way to classify an image for test without having classifier model? I meant if I am working with 1M celeb dataset, I need to train classifier as it mentioned in wiki to get first the model (.pkl) for classifier then use it?

[Zhou-Yicheng]

need to train classifier as mentioned in wiki to get first the model (.pkl) for classifier then use it?

I'm afraid so