search

TimDettmers / sparse_learning

Sparse learning library and sparse momentum resources.
MIT License
377 stars 45 forks source link

Not getting desired accuracy on CIFAR-100 #12

Closed Puneet2000 closed 3 years ago

Puneet2000 commented 5 years ago

I am getting around 75% accuracy on ResNet34 on CIFAR100. Hyperparameters are default.

TimDettmers commented 5 years ago

Thanks for opening an issue on this. From the PyTorch CIFAR-100 repo I find that the performance should be around 78%. Default parameters means 5% weights which is probably not enough to replicate dense performance. From my experience with ResNets you will need about 30% weights to replicate dense performance. The learning rate schedule might also be a bit off. If you want to get better performance, you should also try to tune the learning rate. The training curve differs slightly between sparse and dense networks.

Puneet2000 commented 5 years ago

Thanks, @TimDettmers!