search

changlin31 / BossNAS

(ICCV 2021) BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
137 stars 20 forks source link

How to obtain the searched model #11

Open hamdjalil opened 2 years ago

hamdjalil commented 2 years ago

I used the searching code for a small number of epochs, can you share where exactly is the best model architecture stored when any custom NAS is performed? the pth files are saved in work_dir but im not sure where the corresponding architecture is stored so I can use a custom generated model together with these weights ?

changlin31 commented 2 years ago

Hi, @hamdjalil

BossNAS is a "two-stage" NAS method, i.e., the searched model need to be retrained from scratch. We do not support loading the weights of the searched model from the supernet. But I expect that will accelerate training. We have implemented .dispatch() method, which can detach a path from the supernet. But I am not sure if this method would work properly, as the whole BossNAS pipeline do not use this method. If you need to use the searched model directly, you could try and modify this .dispatch() method. https://github.com/changlin31/BossNAS/blob/bb8b26a09e3ae7889dd118950b022be6b3ce99a0/searching/bossnas/models/supernets/mbconv_supernet.py#L203

"path rank" files are not checkpoints; they are architecture rankings. If you want to perform the original BossNAS pipeline, you need to construct and retrain the top architecture.

hamdjalil commented 2 years ago

"path rank" files are not checkpoints; they are architecture rankings. If you want to perform the original BossNAS pipeline, you need to construct and retrain the top architecture.

Can you please share how to construct and retrain the top architecture from the original BossNAS pipeline? My goal is to ultimately use a custom dataset and perform NAS to obtain a model using the BossNAS pipeline.

sekhaish commented 2 years ago

"path rank" files are not checkpoints; they are architecture rankings. If you want to perform the original BossNAS pipeline, you need to construct and retrain the top architecture.

Can you please share how to construct and retrain the top architecture from the original BossNAS pipeline? My goal is to ultimately use a custom dataset and perform NAS to obtain a model using the BossNAS pipeline.

Hi @hamdjalil, were you able to construct the top Architecture for retraining? If not Could you help me with this? @changlin31 Thank you for your time in advance.