search

facebookresearch / vissl

VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.
https://vissl.ai
MIT License
3.25k stars 331 forks source link

[WIP] Byol Repro (v2) #454

Open iseessel opened 2 years ago

iseessel commented 2 years ago

Continues work done by @lluiscastrejonsubira.

Further Improvements and bug fixes by @iseessel and @pranavsinghps1.

We are still unable to reproduce the full results of the BYOL paper. Here is our current linear transfer results. That is running byol_8node_resnet.yaml and then running /eval_resnet_8gpu_transfer_in1k_linear.yaml and reporting the best test results:

top_1: 69.726
top_5: 89.408

However, the BYOL paper reports the following, so we are ~3% lower. 

top_1: 72.5
iseessel commented 2 years ago

Note this replaces: @lluiscastrejonsubira PR here: https://github.com/facebookresearch/vissl/pull/290

facebook-github-bot commented 2 years ago

@iseessel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot commented 2 years ago

@iseessel has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot commented 2 years ago

@iseessel has updated the pull request. You must reimport the pull request before landing.

zhifanwu commented 2 years ago

@iseessel, thanks for your great work. Can I know what accuracy does this version of BYOL achieve now?

iseessel commented 2 years ago

@zhifanwu This will be finished in the next month or so. I believe it is currently fully reproduced, but will want to run some more experiments to verify.

YannDubs commented 1 year ago

Hi @iseessel are there any updates on that? Will you upload the weights to the model zoo?