search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.42k stars 186 forks source link

SimSiam results on ImageNet. #16

Closed poodarchu closed 3 years ago

poodarchu commented 3 years ago

Can you share your results on the full Imagenet-1k dataset? I think the results on imagenet-100 is not sufficient to prove that the implementation is right.

My implementation here can reach 65% top-1 acc on ImageNet 1k (using mocov2's linear evaluation protocol), https://github.com/poodarchu/SelfSup/blob/master/examples/simsiam/SimSiam.res50.imagenet.256bs.224size.100e.lin_cls/README.md, as reported in the paper, this number should be ~66.7%。

Moreover, my results on CIFAR can match the performance in paper exactly.

vturrisi commented 3 years ago

I've previously tested in cifar10 (800 epochs) without a static lr for the projection head and without zero_init_residual and it converged to 91.6% acc.

I'm currently running on imagenet with both tricks implemented and I'll update the repo as soon as it's done.

poodarchu commented 3 years ago

I've also reach the same performance as the paper on CIFAR 10, but it may be not the case as for Imagenet-1K

poodarchu commented 3 years ago

I update my results of SimSiam here: https://github.com/poodarchu/SelfSup/blob/master/examples/simsiam/SimSiam.res50.imagenet.256bs.224size.100e.lin_cls/README.md 100 epoch on ImageNet-1K can reach 67.6% (paper 67.7%, without constant lr in MLP head).