search

microsoft / esvit

EsViT: Efficient self-supervised Vision Transformers
MIT License
408 stars 45 forks source link

Unable to reproduce the KNN results #8

Closed kikacaty closed 3 years ago

kikacaty commented 3 years ago

Hi, I am trying to reproduce the knn results but fail to do so. I am using the pretrained model from the checkpoint on ImageNet-1K following the script provided.

I got the following results:

10-NN classifier result: Top1: 1.876, Top5: 3.462
20-NN classifier result: Top1: 1.872, Top5: 3.912
100-NN classifier result: Top1: 1.85, Top5: 4.884
200-NN classifier result: Top1: 1.834, Top5: 5.352

Is there any chance that the model checkpoint is incorrect?

Thanks!

ChunyuanLI commented 3 years ago

I did a quick test, it works well. Make sure that you load the checkpoint correctly. Please see the message in blue box to ensure the backbone is loaded. Thanks.

image

kikacaty commented 3 years ago

Thanks for the quick response. I verified that I did load the backbone correctly.

I noticed that your data loader consists of only 10k train images while I am using the entire 1200k images. Could that be the issue?

Thanks,

ChunyuanLI commented 3 years ago

Are you able to resolve the issue? One possible reason might be that your labels are not well aligned.