Transfer learning on custom dataset (Asian faces)?

[taewookim]

This is a follow up to this thread

I'm not talking about custom classifier from @davidsandberg (i.e. training classifier on known classes).. this problem im referring to is mainly calculating L2 distance on Asian faces. I tried all models from past (20170131-005910, 20170131-234652, 20180408-102900, and 20180402-114759)... and the L2 distances for same face seem to be too high and vice versa (different faces have l2 distances too low)

https://s7.postimg.cc/yi28ve5rv/image.png

1) Are there notes / repos / tutorials specifically on transfer learning for custom dataset (particularly Asian faces).

2) Anyone working on this? Looking for collaborators. Would gladly open source this work w/some guidance & help. (Ideally Keras based, but i guess TF would work too)

3) What is the dataset supposed to look like? (i.e. specific angles, labels, etc.) Would appreciate if someone point me to a sample dataset

[Dyex719]

As far as I understand, you want data to fine-tune the pre-trained model to perform well on asian faces. I know of this dataset, http://cvit.iiit.ac.in/projects/IMFDB/ . Maybe this will help you.

[rain2008204]

@taewookim have you solve this problem? can you share with me ?

[taewookim]

mmm it's indian face dataset was looking for more east asian dataset

@rain2008204 : not yet, still searching for asian face dataset

[caocuong0306]

I guess this is the one you're looking for: Asian-Celeb 93,979 ids with 2,830,146 aligned images. https://github.com/deepinsight/insightface/issues/256

Hope to hear your progress soon. Good luck.

[hungnv21292]

Hi @taewookim, @caocuong0306 Can you share the progress of finetune the Asian face datasets? I also want to finetune it. :)

Thanks you so much.

[hungnv21292]

hi all, I also finetune with an asian face datasets use tripletloss method with pretrained model is model-20180402-114759.ckpt-275 and get an error: "InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [128] rhs shape= [512] [[Node: save/Assign_20 = Assign[T=DT_FLOAT, _class=["loc:@InceptionResnetV1/Bottleneck/BatchNorm/moving_variance"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](InceptionResnetV1/Bottleneck/BatchNorm/moving_variance, save/RestoreV2/_1239)]]"

I also try to fix follow comment as in #817 "all_vars = tf.trainable_variables() var_to_restore = [v for v in all_vars if not v.name.startswith('Logits')] saver = tf.train.Saver(var_to_restore, max_to_keep=5,keep_checkpoint_every_n_hours=1.0)"

Please help me if everybody have any solution for this problem. Thank you so much.

[hungnv21292]

Hi @caocuong0306 , I have use Asian Celebrity Face from http://trillionpairs.deepglint.com/overview This dataset is clean or not. This dataset have overlap with LWF or any other datasets?

Thanks you

[zeinabfarhoudi]

Hi I need help to transfer learning on my own dataset. I have an error when running train_softmax.py by giving --pretrained_model as an argument

[hungnv21292]

Hi @farhodi

Can you show the error log when you run the training?

[hungnv21292]

Hi all,

In Wiki of @davidsandberg have public two model weights. I have some concerns.

• Both of public the model weights have embedding size = 512?
• Where can I download the model weights with embedding size = 128?

Thank you so much.

[Dyex719]

Both the top performing models (trained on VGGface and Casia Webface) have 512 embedding size. If you really need a 128 embedding size model, you can retrain the net and mention that you want a 128 embedding size.

[xlphs]

Hello,

I'm finetuning for Asian faces and got good results after 100 epochs, but I don't think the network has converged. The finetuned model gives significantly lower cosine similarity on different faces in my test set. I train the entire network with a smaller learning rate. Still experimenting with lr, I'd appreciate if someone could share their experience on this.

For dataset, I used 8k identities from the deepglint asian celebrity dataset, plus 300 from vggface2 to balance things a bit. LFW accuracy dropped to 98%.

[rain2008204]

@xlphs just only use more identities dataset. It's important that more identities than the number of average person. I used more than 10K identities. On my view, I'm finetuing for more than 500epochs.

[xlphs]

@rain2008204 Thank you, I'll add more identities and continue training.

[xlphs]

Issue #723 is very helpful. You can change train_softmax.py to use arcface loss and continue training on the pretrained model. It actually converges very fast so be careful not to overfit, I simply pick the epoch that gives highest LFW accuracy, the best I got is 99.55%. This method is the best imho because it's very fast to train.

[angervision]

Issue #723 is very helpful. You can change train_softmax.py to use arcface loss and continue training on the pretrained model. It actually converges very fast so be careful not to overfit, I simply pick the epoch that gives highest LFW accuracy, the best I got is 99.55%. This method is the best imho because it's very fast to train.

@xlphs Could you elaborate more on that one, please? Do you mean that you can routinely continue training the existing model with the new batches of faces (say 5k per additional train) after performing the changes that you've mentioned above? I would be happy to see an example of the code\passing parameters to train_softmax.py if that's possible. Thanks a lot.

[xlphs]

@angervision You can read the wiki page https://github.com/davidsandberg/facenet/wiki/Training-using-the-VGGFace2-dataset . Basically the answer is yes. To continue training use --pretrained_model, and since you probably use a different number of identities, you have to create a new saver that restores model like this:

all_vars = tf.trainable_variables()
var_to_restore = [v for v in all_vars if not v.name.startswith('Logits')]
new_saver = tf.train.Saver(var_to_restore)

# ...
# later, use this new_saver to restore pretrained model
if pretrained_model:
    print('Restoring pretrained model: %s' % pretrained_model)
    new_saver.restore(sess, pretrained_model)

Of course the tensor name (Logits) may be different when using a different loss.

[cp0000]

Hi @caocuong0306 , I have use Asian Celebrity Face from http://trillionpairs.deepglint.com/overview This dataset is clean or not. This dataset have overlap with LWF or any other datasets?

Thanks you

hi,i have used the celebrity-asian datasets to finetune the vgg2face pretained model.what's your last loss, my loss is 4.1,it's really high.

[Sunwc0]

hi all, I also finetune with an asian face datasets use tripletloss method with pretrained model is model-20180402-114759.ckpt-275 and get an error: "InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [128] rhs shape= [512] [[Node: save/Assign_20 = Assign[T=DT_FLOAT, _class=["loc:@InceptionResnetV1/Bottleneck/BatchNorm/moving_variance"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:GPU:0"](InceptionResnetV1/Bottleneck/BatchNorm/moving_variance, save/RestoreV2/_1239)]]"

I also try to fix follow comment as in #817 "all_vars = tf.trainable_variables() var_to_restore = [v for v in all_vars if not v.name.startswith('Logits')] saver = tf.train.Saver(var_to_restore, max_to_keep=5,keep_checkpoint_every_n_hours=1.0)"

Please help me if everybody have any solution for this problem. Thank you so much.

It seems to me that 512D embeddings contains some less informative features, which are neglected, and even help if you use SVM for face verification, but adds a lot of 'noise' if you try to compute cosine/euclidean distance between two embeddings.

[Zumbalamambo]

@cp0000 my loss is 0.043 with the deepglint Asian dataset.

[LeeTaiTai]

@Zumbalamambo yout loss is 0.043, and how much is your train epoches?, i trained Epoch: [258][528/1000] Time 0.335 Loss 2.395 Xent 0.951 RegLoss 1.443 Accuracy 0.911 Lr 0.00050 Cl 0.356, do you use center loss？