How you define Limited Computational Resources?

[khawar-islam]

Hi @pengyuLPY

Thank you for your wonderful work. I would like to know that how you define the term like "Limited Computational Resources". Is this term depending on your hardware configuration? Do you restrict your hardware configuration?

Regards, Khawar

[pengyuLPY]

I appreciate your interest in our work. In this work, we propose a method (Virtual-FC layer) to train a large-scale face recognition dataset with limited computational resources, which means the dataset can be trained even with a single GPU. Without our Virtual-FC, the dataset with millions of identities is challenging to be trained because the large-scale classification layer (millions of neurons) costs lots of computational resources (e.g., GPU memories and FLOPs). The Virtual-FC layer tries to reduce the computational consumptions significantly (hundreds of times). Our proposal does not depend on the specific hardware configuration. However, we think it will work well in all the hardware. Surely, the cost of the computational resource (the number of GPUs) and the performance should be balanced. By the way, our Virtual-FC layer can be accelerated by Data-Parallel if multiple GPUs are available, but it still works with a single one.

[khawar-islam]

@pengyuLPY Thank you for your clarification. would it be possible to replace nn.Conv2d or nn.Linear to your layer?

[pengyuLPY]

Yes, of course. Our Virtual-FC layer is improved from the FC layer (nn.Linear in PyTorch). You can replace it with layers.

[khawar-islam]

Thank you @pengyuLPY . In my network HEAD, I have implemented an ArcFace loss function as a classification HEAD for the input features and number of classes. Would it be possible for you to share the simplest example of your code? please, If you point out the python file that contains your FC layer and arcFace code. Thank you

Like, https://github.com/pengyuLPY/Virtual-Fully-Connected-Layer/blob/master/CNNs/resnet_identityMapping_face.py

There are two FC layers which one is correct?

self.fc = ArcMarginProduct(feature_len, num_classes_1, m = 0.5)
self.fc2 = DictArcMarginProduct(feature_len, out_features=num_classes_2, out_features_test=num_classes_2, label_dict=label_dict, m = 0.5)

[khawar-islam]

Thank you. It works very well

[pengyuLPY]

Perfect^_^ You are really welcome.

Best Wishes