search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.56k stars 281 forks source link

[NAS-bench-201] Questions of applying GDAS on CIFAR-100 #64

Closed GongXinyuu closed 4 years ago

GongXinyuu commented 4 years ago

Hi Xuanyi, Thanks for your great work! I've encountered some problems when I try to reproduce the result. I'd really appreciate it if you can spare some time on it.

Describe the bug Cannot reproduce the search result of GDAS on CIFAR100

To Reproduce

bash ./scripts-search/algos/GDAS.sh cifar100 1 777
bash ./scripts-search/algos/GDAS.sh cifar100 1 888
bash ./scripts-search/algos/GDAS.sh cifar100 1 999

Ubuntu, Python 3.6, pytorch 1.4.0

Expected behavior Should match the number reported in the paper.

Screenshots I run the script many times, but I got the best accuracy of 67.56 on CIFAR-100. The number is obtained on 'ori-test' set.

D-X-Y commented 4 years ago

In our NAS-Bench-201 paper, we searched on CIFAR-10, and report the results of CIFAR-10 searched architecture on three datasets. We did not do the above experiments in our paper.

GongXinyuu commented 4 years ago

Thanks for your prompt clarification!