search

idstcv / ZenNAS

223 stars 37 forks source link

The setting for num param=0.5m #31

Closed dzk9528 closed 1 year ago

dzk9528 commented 1 year ago

Hi. I am interesting in what is the parameter setting for num param=0.5m

budget_model_size=5e5
max_layers=18
population_size=512
evolution_max_iter=480000  # we suggest evolution_max_iter=480000 for

Is it like this one?

MingLin-home commented 1 year ago

You can find all model searching parameters here: https://github.com/idstcv/ZenNAS/tree/main/scripts

dzk9528 commented 1 year ago

Thanks for the reply, but main/scripts only contain 1M parameters for ZenNAS and I also found that in the table you have 0.5M parameters result. So I wonder what is the searching setting for the 0.5M parameters, which is helpful since I currently worked on model for edge devices.

MingLin-home commented 1 year ago

You might use the same hyper-parameter of 1.0M for 0.5M. You could also slightly reduce the number of max layers (say, 0.8x) to see the impact.

In case you might be interested, please also check our most up-to-date work DeepMAD on CVPR 2023 (https://arxiv.org/abs/2303.02165). The algorithm is implemented in https://github.com/alibaba/lightweight-neural-architecture-search.

dzk9528 commented 1 year ago

Thanks for the reply and I will try it. Already have 1M result, 96.88 on cifar10.