A question about the meaning of a specific code block

[wmingdao]

Hi, I have some questions about the meaning of the code block in ./moco/builder_inter.py Row 79-87, as follows:

    `if mlp:  # hack: brute-force replacement
        dim_mlp = self.encoder_q.fc.weight.shape[1]
        self.encoder_q.fc = nn.Sequential(nn.Linear(dim_mlp, dim_mlp), nn.ReLU(), self.encoder_q.fc)
        self.encoder_k.fc = nn.Sequential(nn.Linear(dim_mlp, dim_mlp), nn.ReLU(), self.encoder_k.fc)`

       `dim_mlp_2 = self.encoder_r.fc.weight.shape[1]
        self.encoder_r.fc = nn.Sequential(nn.Linear(dim_mlp_2, dim_mlp_2), nn.ReLU(), self.encoder_r.fc)
        self.encoder_l.fc = nn.Sequential(nn.Linear(dim_mlp_2, dim_mlp_2), nn.ReLU(), self.encoder_l.fc)`

Why not define a two-layer fc directly during initializing the model for the encoder_q/encoder_k? In the source code, you first define single layer fc, and here you try define the two-layer fc by self.encoder_q.fc = nn.Sequential(nn.Linear(dim_mlp, dim_mlp), nn.ReLU(), self.encoder_q.fc). I find the same operation is done in the source code of MOCO. Maybe it is a simple question, but as a novice, I look forward to hearing from you. Thanks!

[fmthoker]

Hey, you are right, we can directly define a two-layer fc during initialization of the encoder_q/k. It would not make any difference since the original fc layer is overwritten.