Paper Implementation

[khawar-islam]

Dear @tiandunx

Thank you for your work. In the Virtual FC paper at CVPR 2021, we have to change the network HEAD and replace it with the Virtual FC layer which takes two parameters in_feat and out_feat. Is your work like them? because I saw your code and didn't get any clear definition. If possible for you, please mention some small examples, because I want to use your layer in my network.

Regards, Khawar

[tiandunx]

In our work, HEAD is just DCP. The backbone is defined in resnet_def/*.py. We only use backbone to extract feature which is later used to do matrix product with dcp.

[khawar-islam]

Right, Like in the Virtual FC layer at CVPR 2021, the code looks like this. You have a function and you have to provide in_feat and out_feat. Your implementation is quite confusing for me, if you indicate me a function in your code then it would easy for the researcher to implement your method.

class ArcMarginProduct(nn.Module):
    def __init__(self, in_features=128, out_features=200, s=32.0, m=0.50, easy_margin=False):
        super(ArcMarginProduct, self).__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.s = s
        self.m = m
        self.weight = Parameter(torch.Tensor(out_features, in_features))
        nn.init.xavier_uniform_(self.weight)
        # init.kaiming_uniform_()
        # self.weight.data.normal_(std=0.001)

        self.easy_margin = easy_margin
        self.cos_m = math.cos(m)
        self.sin_m = math.sin(m)
        # make the function cos(theta+m) monotonic decreasing while theta in [0°,180°]
        self.th = math.cos(math.pi - m)
        self.mm = math.sin(math.pi - m) * m

    def forward(self, x, label, testing=False):
        cosine = F.linear(F.normalize(x), F.normalize(self.weight))
        sine = 1.0 - torch.pow(cosine, 2)
        sine = torch.where(sine > 0, sine, torch.zeros(sine.size(), device='cuda'))
        sine = torch.sqrt(sine)
        # sine = torch.sqrt(1.0 - torch.pow(cosine, 2))
        phi = cosine * self.cos_m - sine * self.sin_m
        if self.easy_margin:
            phi = torch.where(cosine > 0, phi, cosine)
        else:
            phi = torch.where((cosine - self.th) > 0, phi, cosine - self.mm)

        one_hot = torch.zeros(cosine.size(), device='cuda')
        one_hot.scatter_(1, label.view(-1, 1).long(), 1)
        if testing == False:
            output = (one_hot * phi) + ((1.0 - one_hot) * cosine)
            output *= self.s
            return output
        else:
            output0 = (one_hot * phi) + ((1.0 - one_hot) * cosine)
            output00 = output0 * self.s

            output1 = cosine
            output11 = output1 * self.s
            return output00, output11

I also tried to use ffc_ddp.py file but it gives an error

ModuleNotFoundError: No module named 'lru_utils'

[tiandunx]

@khawar512 You can simply rename lru_python_impl.py to lru_utils.py so you can fix ModuleNotFoundError. For ArcMarginProduct, member function add_margin in ffc_ddp.py plays the same role as ArcMarginProduct. At line 34 in ffc_ddp.py "self.register_buffer('queue', torch.rand(2, queue_size, feat_dim))" is the weight.

[khawar-islam]

@tiandunx thank you for your reply. I have fixed all the problems and just one problem is remaining. In your code, I am using your FFC layer like below, as same previous implementation..

            elif self.loss_type == 'ArcFace':
                self.loss = ArcFace(in_features=embed_dims[3], out_features=num_classes, device_id=self.GPU_ID)
            elif self.loss_type == 'SFace':
                self.loss = SFaceLoss(in_features=embed_dims[3], out_features=num_classes, device_id=self.GPU_ID)
            elif self.loss_type == 'ArcMarginProduct':
                self.loss = ArcMarginProduct(in_features=embed_dims[3], out_features=num_classes)
            elif self.loss_type == 'FFC':
                self.loss = FFC(feat_dim=embed_dims[3], num_class=num_classes)

Your FFC takes in_features and the number of classes, but your FFC layer also need network type. I don't want to give network type because before this FFC layer, I already mentioned everything about the network.

How we can get of net_type parameter?

Traceback (most recent call last):
  File "/media/khawar/HDD_Khawar/facerectransformer-main/code/train.py", line 236, in <module>
    in_chans=3
  File "/media/khawar/HDD_Khawar/facerectransformer-main/code/vit_pytorch/pvt_v2.py", line 338, in __init__
    self.loss = FFC(feat_dim=embed_dims[3], num_class=num_classes)
TypeError: __init__() missing 1 required positional argument: 'net_type'

[tiandunx]

@khawar512 FFC is not a loss layer. It's proposed to deal with the case where id is extremely huge and cannot be fed into a single gpu while distributed partial fc is time-consuming and costly. So the class FFC encapsules backbone and weight simultaneously.

[tiandunx]

@khawar512 net type is the backbone you use like r50, r100, efficient net and other type of networks etc.

[khawar-islam]

Dear @tiandunx, I understand your function that this is network type, Like in other loss functions, we just need to give the feature dimension and number of classes. Like below, Every function is only taking two parameters, my backbone is completely different and I would like to treat your FC layer as only an FC layer without giving a network name parameter in the FC layer.

https://github.com/ZhaoJ9014/face.evoLVe/blob/master/head/metrics.py

Implementation, please search loss_type, so that you can get some idea https://github.com/zhongyy/Face-Transformer/blob/main/copy-to-vit_pytorch-path/vits_face.py