Some errors in the network.py

[MTandHJ]

First is clear that some args or kwargs don't match, for instance,

def resnext50_32x4d(pretrained=False, progress=True, **kwargs):
    r"""ResNeXt-50 32x4d model from
    `"Aggregated Residual Transformation for Deep Neural Networks" <https://arxiv.org/pdf/1611.05431.pdf>`_
    Args:
        pretrained (bool): If True, returns a model pre-trained on ImageNet
        progress (bool): If True, displays a progress bar of the download to stderr
    """
    kwargs['groups'] = 32
    kwargs['width_per_group'] = 4
    return ResNet(Bottleneck, [3, 4, 6, 3], args, **kwargs)

Besides, in this implement, ResNet has a max pooling operation at first; however, I found a sentence in the SimCLR's paper:

Because CIFAR-10 images are much smaller than ImageNet images, we replace the first 7x7 Conv of stride 2 with 3x3 Conv of stride 1, and also remove the first max pooling operation.

Thus, will this detail make difference?

[AidenDurrant]

Hi there,

You are correct about some of the arguments, this repo is quite messy and was just my first attempt at getting SimCLR working, I will clean it up at some point when I have the time, so my apologies for the messy arguments.

For your second point, I had already addressed this in lines 150-172 in the network.py file to change the stem configuration dependent on the dataset.

if args.dataset == 'cifar10' or args.dataset == 'cifar100' or args.dataset == 'tinyimagenet':

            # CIFAR Stem

            self.stem = nn.Sequential()

            self.stem.add_module('conv0', nn.Conv2d(3, self.inplanes, kernel_size=3, stride=1, padding=1,
                                                    bias=False))
            self.stem.add_module('BN1', norm_layer(self.inplanes))
            self.stem.add_module('ReLU1', nn.ReLU(inplace=True))

        # e.g. ImageNet
        else:

            self.stem = nn.Sequential()

            self.stem.add_module('conv0', nn.Conv2d(3, self.inplanes, kernel_size=7, stride=2, padding=3,
                                                    bias=False))
            self.stem.add_module('BN1', norm_layer(self.inplanes))
            self.stem.add_module('ReLU1', nn.ReLU(inplace=True))
            self.stem.add_module('MaxPool1', nn.MaxPool2d(kernel_size=3, stride=2, padding=1))