DonDominic / MP2Net

The official code of our TGRS'24 paper MP2Net: Mask Propagation and Motion Prediction Network for Multi-Object Tracking in Satellite Videos
MIT License
17 stars 0 forks source link

Why does the network structure in ConvGRUNet use module DeConvGRU? #11

Open ciomphzx opened 1 day ago

ciomphzx commented 1 day ago

Dear author, Thank you for your outstanding work! While reading the source code, I came across a question: the paper mentions two GRUNets for encoding and decoding, but in the source code, both the ConvGRUNet and DeConvGRUNet classes use DeConvGRU units. Although there is an implementation of ConvGRU in the source code, it is not used. Could you clarify if these two modules have the same functionality?

(This refers to lines 437 and 440 in lib\models\dladcn_gru.py.)

class ConvGRUNet(nn.Module):
    def __init__(self):
        super(ConvGRUNet, self).__init__()
        self.conv_gru_layer1 = DeConvGRU(3, 16, (7, 7), 1)
        self.maxpool1 = nn.MaxPool3d(kernel_size=(3,3,3), stride=(1,1,1), padding=(1,1,1))

        self.conv_gru_layer2 = DeConvGRU(16, 16, (3, 3), 1)

    def forward(self, c0_sta):
        # layer 1
        c0_seq, _ = self.conv_gru_layer1(c0_sta)                    # B, N, 32, 512, 512
        c0_seq_down = self.maxpool1(c0_seq.transpose(1, 2)).transpose(1, 2)     # B, N, 32, 256, 256

        # layer 2
        c1_seq, _ = self.conv_gru_layer2(c0_seq_down)               # B, N, 64, 256, 256

        return c1_seq
class DeConvGRUNet(nn.Module):
    def __init__(self):
        super(DeConvGRUNet, self).__init__()
        self.proj0 = nn.Sequential(
            nn.Conv2d(32, 16, kernel_size=3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(16)
        )
        self.proj_hm = nn.Sequential(
            nn.Conv2d(16, 16, kernel_size=3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(16)
        )
        for m in self.modules():
            if isinstance(m, nn.BatchNorm2d):
                nn.init.constant_(m.weight, 1)
                nn.init.constant_(m.bias, 0)

        self.deconv_gru_layer = DeConvGRU(16, 16, (3, 3), 1)

I retrained the model using the ICPR dataset and obtained the following metrics. However, the detection metrics for the sequence image with index 003 are abnormal. Could you please explain the possible reasons? image Looking forward to your response.

ciomphzx commented 1 day ago

the detection metrics for the sequence image with index 003 are abnormal. I checked the sequence images with the number 003, and the number of targets in the images is very small.