search

ayulockin / SwAV-TF

TensorFlow implementation of "Unsupervised Learning of Visual Features by Contrasting Cluster Assignments".
https://app.wandb.ai/authors/swav-tf/reports/Unsupervised-Visual-Representation-Learning-with-SwAV--VmlldzoyMjg3Mzg
Apache License 2.0
85 stars 12 forks source link

[QUESTION] Difference 10-epochs, 40-epochs #9

Closed RGring closed 3 years ago

RGring commented 3 years ago

Hi Ayush and Sayak, I read your blog on wandb and find it interesting to see the performance on a relatively small dataset. Great work!

I am trying to reproduce your results with the official swAV-implementation. Though I only reach the 10-epochs plateau (~2.7).

I was wondering what are the difference between both runs? I found the following, did I miss any?

Just curious, if you achieved the same results with the original pytorch implementation?

Thanks in advance :)

sayakpaul commented 3 years ago

Thank you for your interest @RGring.

Here are a couple of suggestions that the authors have put in the official repository and we have verified that ourselves and they work! We are yet to publish the changes but just wanted to confirm that those suggestions actually work.

Just curious, if you achieved the same results with the original pytorch implementation?

Actually, we did not run the official implementation so won't be able to lament.

I hope this helps.

RGring commented 3 years ago

Thanks for your answer. I've seen the suggestions and some of them helped, indeed. I guess, now it is a matter of hyperparameter-tuning. Looking forward to your new best score on the flower dataset :).

Thanks again!

sayakpaul commented 3 years ago

Yes, that is generally the case. Sometimes simplifying the data augmentation pipeline also helps quite a lot.

You can also take a look at SimSiam (https://github.com/sayakpaul/SimSiam-TF) which I open-sourced yesterday. It's by FAIR. It's way simpler and it provides pretty comparable results to SwAV, BYOL, MoCov2, and SimCLR.