Training with ArcFace Loss

[neklom]

Hi ! Did anyone trained the model using arcface loss and using the same code in train_softmax.py ?

I trained a model using the ArcFace loss using this bit of code :

prelogits, _ = network.inference(image_batch, args.keep_probability, 
            phase_train=phase_train_placeholder, bottleneck_layer_size=args.embedding_size, 
            weight_decay=args.weight_decay)
embeddings = tf.nn.l2_normalize(prelogits, 1, 1e-10, name='embeddings')
w_init_method = slim.initializers.xavier_initializer()
logits = arcface_loss(prelogits, label_batch, len(train_set), w_init_method)
arcface_loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=label_batch)
arcface_loss_mean = tf.reduce_mean(arcface_loss, name='arcface_loss')

pred = tf.nn.softmax(logits)
correct_prediction = tf.cast(tf.equal(tf.argmax(pred, 1), tf.cast(label_batch, tf.int64)), tf.float32)
accuracy = tf.reduce_mean(correct_prediction)

So I added this line of code before calculating the accuracy : pred = tf.nn.softmax(logits)

I used almost the same hyper-parameters used in this repo for training with softmax (From learning rate schedule, etc ...).

The problem is that I get an accuracy of 0.9883 on LFW and a validation rate = 0.9643 @ FAR=0.001 , but I get poor results on the training/validation set. I got around 53% accuracy on training and around 38% on testing set. The RegLoss is ~3.270 and the arcface loss is between 5.0 and 6.5

Can anyone explain me the problem ? And is it normal to have such accuracies on train/test sets and get those results on the LFW ?

[jjsjunior]

Hi @neklom did you managed to solve the problem? What dataset did you use? I'm having a hard time trying to use ArcFace from repo auroua. I'm using a subset of Casia with 1k classes just to check that model is converging. But until now the acc of trainset is 0 even after 20 epochs. The loss starts at 20 and saturates at 11. Thanks.

[Marlowess]

Hi ! Did anyone trained the model using arcface loss and using the same code in train_softmax.py ?

I trained a model using the ArcFace loss using this bit of code :

prelogits, _ = network.inference(image_batch, args.keep_probability, 
            phase_train=phase_train_placeholder, bottleneck_layer_size=args.embedding_size, 
            weight_decay=args.weight_decay)
embeddings = tf.nn.l2_normalize(prelogits, 1, 1e-10, name='embeddings')
w_init_method = slim.initializers.xavier_initializer()
logits = arcface_loss(prelogits, label_batch, len(train_set), w_init_method)
arcface_loss = tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=label_batch)
arcface_loss_mean = tf.reduce_mean(arcface_loss, name='arcface_loss')

pred = tf.nn.softmax(logits)
correct_prediction = tf.cast(tf.equal(tf.argmax(pred, 1), tf.cast(label_batch, tf.int64)), tf.float32)
accuracy = tf.reduce_mean(correct_prediction)

So I added this line of code before calculating the accuracy : pred = tf.nn.softmax(logits)

I used almost the same hyper-parameters used in this repo for training with softmax (From learning rate schedule, etc ...).

The problem is that I get an accuracy of 0.9883 on LFW and a validation rate = 0.9643 @ FAR=0.001 , but I get poor results on the training/validation set. I got around 53% accuracy on training and around 38% on testing set. The RegLoss is ~3.270 and the arcface loss is between 5.0 and 6.5

Can anyone explain me the problem ? And is it normal to have such accuracies on train/test sets and get those results on the LFW ?

What training set did you use?