Sweet Gradient Matters: Designing Consistent and Efficient Estimator for Zero-shot Architecture Search (Neural Networks 2023)

This is an official pytorch implementation for "Sweet Gradient Matters: Designing Consistent and Efficient Estimator for Zero-shot Architecture Search".

Environment Requirements

• Python 3.6
• PyTorch 1.8.0

Data Preparation

The final data format is as follows:

Here, we provide detailed data acquisition links:

• cifar10
• cifar100
• ImageNet16-120
• NAS-Bench-Data/all_graphs.json
• NAS-Bench-Data/nasbench1_accuracy.p
• NAS-Bench-Data/NAS-Bench-201-v1_0-e61699.pth

Consistency Experiments

To verify the consistency of the experimental results for NAS-Bench-101, please run:

bash script/Consistency-NB-101.sh

To verify the consistency of the experimental results for NAS-Bench-201, please run:

bash script/Consistency-NB-201.sh

Search Experiments

To verify the search experimental results of NAS-Bench-201, please run:

bash script/Search-NB-201.sh

Citation

Please cite our paper if you find anything helpful.

@article{YANG2023237,
title = {Sweet Gradient matters: Designing consistent and efficient estimator for Zero-shot Architecture Search},
journal = {Neural Networks},
volume = {168},
pages = {237-255},
year = {2023},
issn = {0893-6080},
doi = {https://doi.org/10.1016/j.neunet.2023.09.012},
url = {https://www.sciencedirect.com/science/article/pii/S0893608023005038},
author = {Longxing Yang and Yanxin Fu and Shun Lu and Zihao Sun and Jilin Mei and Wenxiao Zhao and Yu Hu},
}

Acknowledgment

This code is based on zero-cost-nas, AutoDL-Projects. Great thanks to their contributions.