Save the Embeddings from a Complete Triplet Loss Network

[mabedd]

I am working on a Siamese Neural Network with custom Triplet Loss function. As far as I learned from the documentations, we will train a complete network but we want to save only the CNN that is used for embeddings extraction to use it later on with a classifier such as KNN or another.

Network Architeture:

Model: "model_2"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_9 (InputLayer)            [(None, 160, 160, 3) 0                                            
__________________________________________________________________________________________________
input_10 (InputLayer)           [(None, 160, 160, 3) 0                                            
__________________________________________________________________________________________________
input_11 (InputLayer)           [(None, 160, 160, 3) 0                                            
__________________________________________________________________________________________________
sequential_2 (Sequential)       (None, 10)           51144970    input_9[0][0]                    
                                                                 input_10[0][0]                   
                                                                 input_11[0][0]                   
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None,)              0           sequential_2[0][0]               
                                                                 sequential_2[1][0]               
                                                                 sequential_2[2][0]               
==================================================================================================
Total params: 51,144,970
Trainable params: 51,144,970
Non-trainable params: 0
__________________________________________________________________________________________________

CNN Architecture that is used for embeddings extraction:

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_2 (Conv2D)            (None, 159, 159, 64)      832       
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 158, 158, 64)      16448     
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 79, 79, 64)        0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 79, 79, 64)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 399424)            0         
_________________________________________________________________
dense_3 (Dense)              (None, 128)               51126400  
_________________________________________________________________
dropout_4 (Dropout)          (None, 128)               0         
_________________________________________________________________
dense_4 (Dense)              (None, 10)                1290      
=================================================================
Total params: 51,144,970
Trainable params: 51,144,970
Non-trainable params: 0
_________________________________________________________________

I trained the whole network and used base_model.predict()to extract the embeddings. Now , I want to save only the base_model with its learned config during the training. I came across a code on Kaggle through Github Issues that is used for this purpose, I used it and it worked just fine but it gave me a Warning that made me afraid a little bit: [(https://www.kaggle.com/shanmukh05/embeddindgs-extraction-from-siamesemodel)] [(https://github.com/keras-team/keras/issues/8748)]

def copyModel2Model(model_source,model_target,certain_layer="a"):
    i=0        
    for tar,src in zip(model_target.layers,model_source.layers):
        if tar.name==certain_layer:
            break
        if i%2 !=0:
            print(model_source.layers[i].name,i)
            try:
                weights=src.get_weights()
                tar.set_weights(weights)
            except:
                i+=1
                continue
    i+=1  
    print("model source was copied into model target")
    return model_target 

embed_model = copyModel2Model(model,base_model)
embed_model.summary()

embed_model.save('/content/gdrive/MyDrive/RIOTU/projects/snn_upper_face_recognition/tensorflow/colab_outputs/riotu-upper-facenet-tiny.h5')

WARNING:tensorflow:Compiled the loaded model, but the compiled metrics have yet to be built. model.compile_metrics will be empty until you train or evaluate the model.

When I load the model again with load_model I got this warning also: WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually.

Now is this ok or I am missing something while saving the model ? and what is the appropriate way to save a Triplet Loss Network ?

Full Network Code:
def identity_loss(y_true, y_pred):
    return K.mean(y_pred)

def triplet_loss(x, alpha = 0.2):
    # Triplet Loss function.
    anchor,positive,negative = x
    # distance between the anchor and the positive
    pos_dist = K.sum(K.square(anchor-positive),axis=1)
    # distance between the anchor and the negative
    neg_dist = K.sum(K.square(anchor-negative),axis=1)
    # compute loss
    basic_loss = pos_dist-neg_dist+alpha
    loss = K.maximum(basic_loss,0.0)
    return loss

def embedding_model():
  # Simple convolutional model 
  # used for the embedding model.
  model = Sequential()

  model.add(Convolution2D(64, (2, 2), activation='relu',
                        input_shape=(160,160,3)))

  model.add(Convolution2D(64, (2, 2), activation='relu'))
  model.add(MaxPooling2D(pool_size=(2,2)))
  model.add(Dropout(0.25))

  model.add(Flatten())

  model.add(Dense(128, activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(10))

  return model

def complete_model(base_model):
    # Create the complete model with three
    # embedding models and minimize the loss 
    # between their output embeddings
    input_1 = Input((imsize, imsize, 3))
    input_2 = Input((imsize, imsize, 3))
    input_3 = Input((imsize, imsize, 3))

    A = base_model(input_1)
    P = base_model(input_2)
    N = base_model(input_3)

    loss = Lambda(triplet_loss)([A, P, N]) 
    model = Model(inputs=[input_1, input_2, input_3], outputs=loss)
    model.compile(loss=identity_loss, optimizer=Adam(LR))
    return model

[gowthamkpr]

@mohammed-ab99 You can actually save the entire siamese model and load it and extract the layer with desired trained layers and create a new model with a new input layer and the extracted layer. Please take a look at this explanation here. Thanks!

[google-ml-butler[bot]]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler[bot]]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No