search

skhu101 / GM-NAS

Code for our ICLR'2022 paper "Generalizing Few-Shot NAS with Gradient Matching"
MIT License
21 stars 2 forks source link

Could you please provide the source code&weights for reproducing the results of Table 7? #2

Open marsggbo opened 2 years ago

marsggbo commented 2 years ago

Thanks for the great work again!

I've successfully run WS-GM/exp_scripts/rsws-ws-201.sh, and now I obtained four checkpoints of supernets, e.g., supernets_0/1/2/3.pt.

I want to reproduce the results of Table 7, but here are some questions:

  1. what do you mean by top 0.2%/0.5%/1% architectures in NAS-Bench-201? Are these architectures ranked according to their real test accuracy or proxy accuracy?
  2. Table 7 shows that you obtained16 sub-Supernets after partition. However, I only got 4 sub-Supernets.
  3. Could you please provide the source code and model weights for reproducing the spearman score on the top1% models?
ruocwang commented 2 years ago

Hi, sorry for the late reply! I am quite occupied with summer internship lately.

  1. According to their test accuracy.
  2. Table 7 splits 4 times (see #Split column), so there will be 2^4 = 16 sub-supernets.
  3. I think I saved the supernet ckpts somewhere. Please give me some time to locate it and clean up the script. Thanks!