Problem with named_parameters in the encoder

[YuhangMing]

Hi,

I am trying to understand the structure of the network and I found an interesting behaviour of the network.

I was trying to print the "named_parameters()" of the network using the command

for name, val in self.encode.named_parameters():
            print(name, val.shape)

and here is the output I got.

inconv.conv.weight torch.Size([64, 3, 3, 3])
inconv.bn.weight torch.Size([64])
inconv.bn.bias torch.Size([64])
layer1.0.conv1.weight torch.Size([64, 64, 3, 3])
layer1.0.bn1.weight torch.Size([64])
layer1.0.bn1.bias torch.Size([64])
layer1.0.conv2.weight torch.Size([64, 64, 3, 3])
layer1.0.bn2.weight torch.Size([64])
layer1.0.bn2.bias torch.Size([64])
layer1.1.conv1.weight torch.Size([64, 64, 3, 3])
layer1.1.bn1.weight torch.Size([64])
layer1.1.bn1.bias torch.Size([64])
layer1.1.conv2.weight torch.Size([64, 64, 3, 3])
layer1.1.bn2.weight torch.Size([64])
layer1.1.bn2.bias torch.Size([64])
outconv.conv.weight torch.Size([32, 64, 1, 1])
outconv.bn.weight torch.Size([32])
outconv.bn.bias torch.Size([32])

As you can see, there aren't any parameters related to layer2 but when you print the state_dict() of the model, both layer1 and layer2 can be found. This feels like layer2 of the encode module is simply a repeat or an aliase of layer 1 of the encode module. btw I am using pytorch version 1.13.0.

[mbanani]

Hi @YuhangMing,

Yes, layer2 is a repeat for layer 1. This was a bug in the first version of the code and fixing the shared weights actually improves performance (check #5 for more discussion). We chose to keep the code in this form since it matches what's in the paper. Fixing this bug to use two separate layers or even using a larger backbone (eg, SyncMatch uses a modified ResNet18 instead of the five layer ResNet used here) should improve performance.

Let me know if you have any more questions.