Integrating EfficientNet with CenterNet

[chwong1]

I am trying to integrate efficientnet from https://github.com/narumiruna/efficientnet-pytorch/blob/master/efficientnet/models/efficientnet.py.

I added efficientnet.py in models/networks folder and added corresponding code for factory, etc.

Then, I tried to train the network using COCO data, the loss, hm_loss and wh_loss are becoming converged at around 3, 2 and 5 respectively after 140 epoch and cannot reduce anymore. It is not enough to get a good AP result. Do you have any hints to improve?

These codes are added after the last channels in efficientnet by following the code in resnet_dcn.py ` last_channels = _round_filters(1280, width_mult) features += [ConvBNReLU(in_channels, last_channels, 1)]

    self.features = nn.Sequential(*features)
    # self.classifier = nn.Sequential(
    #     nn.Dropout(dropout_rate),
    #     nn.Linear(last_channels, num_classes),
    # )

    # used for deconv layers
    self.inplanes = last_channels
    self.deconv_layers = self._make_deconv_layer(
        3,
        [256, 128, 64],
        [4, 4, 4],
    )

    for head in self.heads:
        classes = self.heads[head]
        # fc = nn.Conv2d(64, classes, 
        #     kernel_size=1, stride=1, 
        #     padding=0, bias=True)
        # fill_fc_weights(fc)
        if head_conv > 0:
            fc = nn.Sequential(
              nn.Conv2d(64, head_conv,
                kernel_size=3, padding=1, bias=True),
              nn.ReLU(inplace=True),
              nn.Conv2d(head_conv, classes, 
                kernel_size=1, stride=1, 
                padding=0, bias=True))
            if 'hm' in head:
                fc[-1].bias.data.fill_(-2.19)
            else:
                fill_fc_weights(fc)
        else:
            fc = nn.Conv2d(64, classes, 
              kernel_size=1, stride=1, 
              padding=0, bias=True)
            if 'hm' in head:
                fc.bias.data.fill_(-2.19)
            else:
                fill_fc_weights(fc)
        self.__setattr__(head, fc)`

[abhigoku10]

@chwong1 can you please share the code base so that even i can train and test it on my end

[chwong1]

@abhigoku10 You may check the code here: https://github.com/motherapp/CenterNet

[Vankeee]

Do you have any progress?

[pierrelzw]

@abhigoku10 You may check the code here: https://github.com/motherapp/CenterNet

Have you test you efficientnet backbone? How about the mAP result?

[sicarioakki]

@pierrelzw I tested CenterNet by integrating your efficientnet.py into the main code base. The training results are very poor with efficientnet backbone. The AP values are all zero even after like 80 epochs, I donot seem to understand what was wrong. I think i have followed everything you told here in this thread.

I have trained with efficientnet_b1 backbone. Your inputs on this would be highly appreciated. @Vankeee @xingyizhou