search

quark0 / darts

Differentiable architecture search for convolutional and recurrent networks
https://arxiv.org/abs/1806.09055
Apache License 2.0
3.92k stars 843 forks source link

parameter "stem-multiplier", "multiplier", "steps" #134

Open xjhcassy opened 4 years ago

xjhcassy commented 4 years ago

Hi, @Catosine, Thanks for sharing the work. I have some quetions about the parameters in model_search.py. I have seen one of your answers in the issues as belows, but I still confused about the parameters "stem-multiplier", "multiplier", "steps", so could you please expland more detail, for example: Have any works found it good in CNN applied to CIFAR/ImageNet. Thank you very much.

I've asked someone who has an research field in automl/nas. He told me that most of the hyperparameters used in DARTS are experience from past, i.e. using stem_mulitiplier=3 is simply because is is found good in previous CNN applied to CIFAR/ImageNet. And that is the same for other hyperparameter you mentioned. He also gives me a suggestion for applying DARTS to other dataset: try to build couple of baselines, and use those hyperparameter as your search space to explore new structures.

Catosine commented 4 years ago

hi @xjhcassy just to be quick: i am NOT the author of this work. But I did play with the code for a while.

Catosine commented 4 years ago

@xjhcassy I also talked to someone in this field: as you have noticed, AutoML is very young and it is not very "auto". i.e. those parameters you mentioned are tuned by human experts.

Catosine commented 4 years ago

@xjhcassy BTW if you are thinking of applied automl to your own task, then it is a very good idea to start as you mentioned in the blackened words.

Catosine commented 4 years ago

@xjhcassy In addition, there are following works of DARTS, such as Progressive DARTS, Partial Channel DARTS, FairDARTS. I think they all on github. Sorry for replying in parts. Good luck:) PF

xjhcassy commented 4 years ago

Thank you very much. I take NAS as my graduation design, so recently I am also playing with the code. After reading many of your answers in the issues, it helps me a lot. Thanks again and good luck!