What kind of feature that Keras take to make a face prediction?

[flyingduck92]

Hello guy, i am studying Face Recognition. And i red this blog post: https://sefiks.com/2018/08/06/deep-face-recognition-with-keras/

I try to make it, and it works. But i still have a few questions in mind:

1. When i encode my image with the pretrained weight from that blog post, i have array with 2622-dimension length. Is this because the last conv layer is 2622?
2. What kind of feature of the face that keras taken for prediction a face? Is it length of eyebrows? eyes? mouth? Nose? Jaw? Because when I put a face image, it produces an array with 2622-dimension length with no further details about what feature that has been extracted to make an array with 2622-dimension.

Thanks.

[flyingduck92]

I was managed to fine tuning VGG16 in Keras and save the Weight

import numpy as np
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Activation
from keras.layers.core import Dense, Flatten
from keras.optimizers import Adam
from keras.metrics import categorical_crossentropy
from keras.preprocessing.image import ImageDataGenerator
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import *
from keras.applications import vgg16

from pathlib import Path

# image_prep
train_path = Path("database") / "train"
test_path = Path("database") / "test"
validation_path = Path("database") / "validation"
class_list = ['P1','P2','P3','P4','P5','P6','P7','P8','P9','P10','P11','P12']

train_batches = ImageDataGenerator().flow_from_directory(train_path, target_size=(224,224), classes=class_list, batch_size=12)
validation_batches = ImageDataGenerator().flow_from_directory(validation_path, target_size=(224,224), classes=class_list, batch_size=4)
test_batches = ImageDataGenerator().flow_from_directory(test_path, target_size=(224,224), classes=class_list, batch_size=12)

# CNN
# ====================================================================================================
# model = Sequential([
#     Conv2D(32, (3,3), activation='relu', input_shape=(224,224,3)),
#     Flatten(),
#     Dense(12, activation='softmax')
# ])
#
# model.compile(Adam(lr=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
# model.fit_generator(train_batches, steps_per_epoch=4,
#                     validation_data=validation_batches, validation_steps=4, epochs=5, verbose=2)

# prediction
# ====================================================================================================
# predictions = model.predict_generator(test_batches, steps=1, verbose=0)
# print(predictions)

# Fine tune VGG16
# ====================================================================================================
vgg16_model = vgg16.VGG16(input_shape=(224, 224, 3))
# vgg16_model.summary()

print(type(vgg16_model))

model = Sequential()
#remove last Dense(1000) layers
for layer in vgg16_model.layers[:-1]:
    model.add(layer)

model.summary()

# set all layers trainable to false
# for layer in model.layers:
#     layer.trainable = False

# add 12 Dense Layer
model.add(Dense(12, activation='softmax'))
model.summary()

# compile fine-tuning vgg16
model.compile(Adam(lr=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit_generator(train_batches, steps_per_epoch=4,
                    validation_data=validation_batches, validation_steps=4, epochs=5, verbose=1)

# save model weight
model.save('vgg16_pretrained.h5')

# summarize history for accuracy
import matplotlib.pyplot as plt
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

# prediction with fine-tuning vgg16
# predictions = model.predict_generator(test_batches, steps=1, verbose=0)
# print(predictions)

To use it in another app I just need to recreate the network and load the Weight

from keras import Sequential, Model
from keras.layers import Dense
from keras.applications import vgg16

def vgg16_finetuning():
    # load vgg-16
    vgg16_model = vgg16.VGG16(input_shape=(224, 224, 3))
    model = Sequential()

    #remove last Dense(1000) layers
    for layer in vgg16_model.layers[:-1]:
        model.add(layer)
    # model.summary()

    # add 12 Dense Layer
    model.add(Dense(12, activation='softmax'))

    # load weight vgg16 finetuning
    model.load_weights("C:/Users/w024029h/PycharmProjects/keras_pretrained/vgg16_pretrained.h5")

    # for layer in model.layers:
    #     print('=====')
    #     print(layer.name)
    #     print(layer.get_input_at(0))
    #     print('=====')

    # print(model.layers[-1].output)

    vgg_face_descriptor = Model(inputs=model.input, outputs=model.layers[-1].output)
    return vgg_face_descriptor

print("Loaded model from disk")

load_vgg16 = vgg16_finetuning()
load_vgg16.summary()

I was integrating with dlib face detector as well. with this I can now make integration with dlib face detector

`

[emc5ud]

I'm not going to go too deep here, but I'd recommend that you watch Coursera's course on deep learning to get a foundation in deep learning. It doesn't take long, and can help with these kinds of questions.

1. When i encode my image with the pretrained weight from that blog post, i have array with 2622-dimension length. Is this because the last conv layer is 2622?

If you look at the model description in the article you posted, you can see that the last layers are named "fc-" followed by some number. FC stands for fully connected. You should look into feed forward neural nets (AKA MLP) in order to get an understanding of what's going on here. Basically your model is learning spatial features in different parts of an image, and eventually flattens those features into a list. At the very end you see the long list of 2622 features that your model has learned and computed.

What kind of feature of the face that keras taken for prediction a face? Is it length of eyebrows? eyes? mouth? Nose? Jaw? Because when I put a face image, it produces an array with 2622-dimension length with no further details about what feature that has been extracted to make an array with 2622-dimension.

You are touching on the classic problem of deep neural networks: they are a "black box". Meaning that it is tricky to look inside a network and determine what interpretable features are being learned. Interpreting learned features of a neural net is an active area of research. You can look into https://github.com/marcotcr/lime for a popular method to interpret networks like yours.