search

lucaslie / torchprune

A research library for pytorch-based neural network pruning, compression, and more.
https://people.csail.mit.edu/lucasl/
MIT License
160 stars 24 forks source link

ResNet56 architecture #1

Closed ghost closed 4 years ago

ghost commented 4 years ago

src/provable_pruning/provable_pruning/util/models/cnn/models/cifar/resnet.py

In your code for ResNet on Cifar dataset, block = Bottleneck if depth >=44 else BasicBlock. This point is really confusing.

Are you sure you used Bottleneck not BasicBlock when you were conducting experiments? As far as I know, Bottleneck is designed for ResNets on ImageNet data, like ResNet50. I read your paper recently, and I am reading your code. Could you please explain what's your resnet56's architecture like?

lucaslie commented 4 years ago

Thank your for your interest in our work. You are right that BottleNeck should only be used for ImageNet.

The cifar cnn models are from this code repository which used to have the check for ResNets but not anymore. It seems like it used to be a small bug/inconsistency there.

I will update the code accordingly to reflect the changes.

So in short, we did use BottleNeck architecture for ResNets on CIFAR but in practice it did not make a big difference. ResNet56 has 6.11% error on CIFAR10 as reported here while we report 6.43%.

ghost commented 4 years ago

Thanks for your response.