Can be used for multiple classifications？

[bleakie]

As shown in the title

[biubug6]

Sure! You need slightly modify network and data input.

[bleakie]

But，how can i modify it？

[rydenisbak]

You should add extra channels for classification/regression/landmarks heads and change loss for separate bbox/landmarks regressions for each class and little change classification loss. You can see there how it works on COCO https://github.com/amdegroot/ssd.pytorch/blob/master/layers/modules/multibox_loss.py

[bleakie]

I saw the use of multibox_loss.py, but how to modify another category, because it has no landmark

[piotr-anyvision]

@bleakie @rydenisbak @biubug6

Did anyone manage to make it work with more classes? I modified the class branch to take into account more classes: self.conv1x1 = nn.Conv2d(inchannels,self.num_anchors*num_classes,kernel_size=(1,1),stride=1,padding=0)

However my loss calculation fails because I have more classification outputs but less prior boxes. Should I just increase the amount of prior boxes to match classification outputs number?

[piotr-anyvision]

To answer my question - yes nr of prior boxes must match wiith class head outputs. The problem was in reshaping the ClassHead output according to nr of classes: return out.view(out.shape[0], -1, num_classes)

It is training at least! :)

[bleakie]

I didn't succeed

[3a532028]

@piotr-anyvision I successfully completed with your suggestion. Thank you so much. But I wonder to know. if I have 2 categories. So the index of each classes should be start from 0 ?

like : 0 for apple , 1 for banana. And the annotation will be: apple => [ x1,y1,x2,y2,0 ] banana => [ x1,y1,x2,y2,1 ]

Also setup the num_classes = 3 , for the negative eaxmple. Am I correct?

[enhany]

@3a532028 can you please share your code?

[3a532028]

@3a532028 can you please share your code?

I just edited the num_classes = 3 in the part of "./models/retinaface.py" as the suggetion from @piotr-anyvision like below:

class ClassHead(nn.Module):
    def __init__(self,inchannels=512,num_anchors=3):
        super(ClassHead,self).__init__()
        self.num_anchors = num_anchors
        self.conv1x1 = nn.Conv2d(inchannels,self.num_anchors*num_classes,kernel_size=(1,1),stride=1,padding=0)

    def forward(self,x):
        out = self.conv1x1(x)
        out = out.permute(0,2,3,1).contiguous()

        return out.view(out.shape[0], -1, num_classes)

And deleted the landmarks output in the NN and related operations. And set num_classes = 3 while training.

After whole training process. It went wired when I was testing the inference phase. All the candidates were up to 99% confidence. (include wrong candidates)

I am not sure that I feed the wrong annotations or some other codes I should implement.

[3a532028]

Well,I just realized that I should also modify the multibox_loss.py as mentioned by @rydenisbak : https://github.com/amdegroot/ssd.pytorch/blob/master/layers/modules/multibox_loss.py

And anntation index should start from 1 like bellow: apple => [ x1,y1,x2,y2,1 ] banana => [ x1,y1,x2,y2,2 ]

Because "0" is for the negative samples.

[stevenluzheng123456]

Hi @3a532028 did you rentinaface with multi-lcasses work? I changed everything exactly followed what you mentioned,but it didn't work at all

[rydenisbak]

@stevenluzheng123456 everything that's you need is mmdetection

[408550969]

除了以上这些外，需要在multibox_loss.py 下将conf_t[pos] = 1注释掉，然而光注释会报错，还需要在wider_face.py下将
if (annotation[0, 4]<0): annotation[0, 14] = -1 注释掉。 但是这会导致landmark loss值异常大，注释掉 if (annotation[0, 4]<0): annotation[0, 14] = -1 后，计算landmark loss时，会把landmark值为-1的那些标签也拿来计算。所以简单的方法是再去掉landmark loss 或者也可以不注释conf_t[pos] = 1 ，对于不同类别的标签的landmark为-1时给予不同的值，（例如标签1的landmark为-1时给予-1，标签2的landmark为-1时给予-2............），然后再增加conf_t[pos1] = 2 ......