search

sthalles / SimCLR

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations
https://sthalles.github.io/simple-self-supervised-learning/
MIT License
2.19k stars 458 forks source link

Is it something wrong with the training model for CIFAR-10 experiments? #34

Closed timqqt closed 3 years ago

timqqt commented 3 years ago

Hi,

I find that the ResNet20 model for CIFAR-10 experiments is not fully correct. The head conv structure should be modified (stride=1 and no pooling,) because the image size of CIFAR-10 is very small.

lim0606 commented 3 years ago

For experiments image_size <= 32, I'm guessing you need to modify initial conv1 and maxpool layers following https://github.com/google-research/simclr/blob/3ad6700c1b139ee18e43f73546b7263a710de699/tf2/resnet.py#L551

model = models.resnet50(pretrained=False, num_classes=1) 
model.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False)
model.maxpool = nn.Identity()
timqqt commented 3 years ago

Thanks. I see.

Fanjie Kong

Ph.D. Student in Machine Learning

Electrical and Computer Engineering Department

Duke University

Durham, NC, 27708

Email: @.***

LinkedIn: https://www.linkedin.com/in/fanjie-kong-9330a6196/

From: Jae Hyun Lim @.> Sent: Saturday, June 5, 2021 16:16 To: sthalles/SimCLR @.> Cc: Fanjie Kong @.>; Author @.> Subject: Re: [sthalles/SimCLR] Is it something wrong with the training model for CIFAR-10 experiments? (#34)

I'm guessing you need to modify initial conv1 and maxpool layers following https://github.com/google-research/simclr/blob/3ad6700c1b139ee18e43f73546b7263a710de699/tf2/resnet.py#L551https://urldefense.com/v3/__https://github.com/google-research/simclr/blob/3ad6700c1b139ee18e43f73546b7263a710de699/tf2/resnet.py*L551__;Iw!!OToaGQ!8YKkFAIy4Cjy7YQIlJEq-ZClwGEgxfGKTusnAUeFW4eoDtu7gn5pjYL5r7sQQcs$

model = models.resnet50(pretrained=False, num_classes=1) model.conv1 = nn.Conv2d(3, 64, kernel_size=3, stride=1, padding=1, bias=False) model.maxpool = nn.Identity()

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/sthalles/SimCLR/issues/34*issuecomment-855290356__;Iw!!OToaGQ!8YKkFAIy4Cjy7YQIlJEq-ZClwGEgxfGKTusnAUeFW4eoDtu7gn5pjYL56Q7GaHw$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AIV7O7ZXY55ZDRKG2Y6SZPTTRKA2VANCNFSM44CAOI7Q__;!!OToaGQ!8YKkFAIy4Cjy7YQIlJEq-ZClwGEgxfGKTusnAUeFW4eoDtu7gn5pjYL5k9fbkH0$.