google-research / big_transfer

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.
https://arxiv.org/abs/1912.11370
Apache License 2.0
1.5k stars 175 forks source link

Unable to replicate published benchmark accuracy on cifar10 dataset #59

Open abdul53saifee opened 2 years ago

abdul53saifee commented 2 years ago

I tried to finetune big transfer code for cifar 10 dataset using the colab provided in TF Blog

The dataset was split on 90% training and 10% validation. The steps per epochs were modified as per dataset number of samples. Bit hyper rule were kept as it is without any modification Model used for finetuning was Resnet-m-50x1 and Resnet-m-101x3

Accuracy of model reached maximum of 94.3% for resnet-m-50x1 against the mentioned accuracy of 97.2 Accuracy for resnet-m-101x3 was reaching max of 93.2 against the mentioned accuracy of 98.5 and no further improvement was observed even running for some more epochs.

Accuracy source :- README.md