search

changlin31 / DNA

(CVPR 2020) Block-wisely Supervised Neural Architecture Search with Knowledge Distillation
234 stars 35 forks source link

Supernet and process_potential.py Workflow Clarifications #29

Closed MohanadOdema closed 3 years ago

MohanadOdema commented 3 years ago

Hi,

I just need to verify I understand your workflow correctly which can also make things clearer for futuristic readers.

1) In the first distillation part sh dist_train.sh, the student supernet is defined as having a single cell per block/stage. Because the default block_cfg in the supernet class have a fixed number of layers and the get_all_models function only gets the combinations of a block's fixed number of layers with the number of candidate operations. Thus, in order to do the search for multiple cells/stage, I need to do multiple separate runs on this search script, with each time I modify the number of layers in block_cfg according to the desired width/depth, leading to multiple .yaml files generated per stage.

2) In process_potential.py, The traversal search can work with multiple cells per block by extending the list entries within potential_yaml and layers_cfgs lists to account for other .yaml files generated for different cells in each stage as this example: potential_yaml=[['$1st_cell$/potential-0.yaml', '$2nd_cell$/potential-0.yaml'], ['$1st_cell$/potential-1.yaml'], ..] and layer_cfgs=[[2,4,4,4,4,1],[3,4,4,4,4,1]]

Thanks in advance.

changlin31 commented 3 years ago

Yes, that is totally correct! I will refer this issue in README for others who want to use multiple cell searching. Thank you for the clarifications.