Closed MarSaKi closed 4 years ago
Such distillation is somewhat like fine-tuning and can be done with train_supernet.py.
You could refer to the script finetune.sh to see how to fine-tune or distill a customized structure with our code.
Thanks, I want to know your experiment in fig 6, is "pruning+distill" means?: teacher: "64_64_64_64_64_64_64_64" student: "32_32_32_32_32_32_32_32" "24_24_24_24_24_24_24_24" "16_16_16_16_16_16_16_16" for each point?
I notice your code can support student with structure "16_16_16...._16", "24_24..._24", "3232..._32", but how to distill a student like "16_16_32_16_32_32_16_16"? If I add config_str in distill options, will it work?