search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.56k stars 281 forks source link

How you find the reduction cell? #6

Closed Catosine closed 5 years ago

Catosine commented 5 years ago

Hello,

I'm kind of interested in how you decide the fixed reduction cell. Do you test different reduction cells? How are the performances of those cells?

Thank you!

D-X-Y commented 5 years ago

Thanks for your interest. It is a little bit heuristic. Since directly applying MaxPool2d with stride = 2 will lose some information, we hope to add another branch https://github.com/D-X-Y/GDAS/blob/master/lib/nas/construct_utils.py#L114 to complement the MaxPool operation. Using 1x3 with 3x1 conv is to reduce the complexity of 3x3 convolution, which is also introduced in GoogleNet.

We also tried several other manually designed reduction cells, while this one gives the best performance on CIFAR.