search

vturrisi / solo-learn

solo-learn: a library of self-supervised methods for visual representation learning powered by Pytorch Lightning
MIT License
1.43k stars 186 forks source link

Method used for the results table #221

Closed Pangoraw closed 2 years ago

Pangoraw commented 2 years ago

I was wondering what evaluation method is used for the performance tables in the readme and could not figure it out from the json configuration file.

Thanks for all the work :+1:

vturrisi commented 2 years ago

Hey, All results reported used linear evaluation (where you pretrain the backbone in a self-supervised way, freeze it and then train a linear classifier on top of it using labelled data). For cifar, we report online linear evaluation (the backbone and the linear layer are trained at the same time, but the gradients of the linear layer are not used to update the backbone at all), as we saw that this resulted in slightly better performance. For imagenet/imagenet100, we report both values.

Recently, we added a way to evaluate with knn, so you might also wanna experiment with this. Note that performance will be lower.

Pangoraw commented 2 years ago

Awesome, thanks for the quick and very detailed answer!