search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.57k stars 282 forks source link

Question regarding running given algorithms on Nasbench102 #44

Closed pipilurj closed 4 years ago

pipilurj commented 4 years ago

Thank you very much for this remarkable contribution. I am trying to run the given algorithms on Nasbench102, I noticed that data path has to be given, why is it needed since the benchmark dataset already has all the evaluation results? What should I do if I just want to test those algorithms on Nasbench102? Thank you very much!

D-X-Y commented 4 years ago

Hi, originally, I hope to provide an alternative option to obtain the performance (training and evaluating) instead of getting from API. Indeed, the current script does not need to set the data path. I just simplify the codes of RS, R-EA, REINFORCE, and BOHB. You can try them now (I already remove the data path args):

[7] bash ./scripts-search/algos/R-EA.sh -1
[8] bash ./scripts-search/algos/Random.sh -1
[9] bash ./scripts-search/algos/REINFORCE.sh -1
[10] bash ./scripts-search/algos/BOHB.sh -1

Note that, DARTS/GDAS/SETN/RANDOM-NAS/ENAS still need to set the data path, because they share the parameters of all architectures. But after searching, using our API, you can directly obtain the performance of their searched architecture without re-training them on CIFAR-10 / CIFAR-100 / ImageNet-16-120.

D-X-Y commented 4 years ago

@pipilurj I will temporally close this issue, please feel free to re-open it if you have further questions.

pipilurj commented 4 years ago

Ok！ Thank you very much for your reply!

D-X-Y notifications@github.com于2020年1月1日 周三上午9:56写道：

@pipilurj https://github.com/pipilurj I will temporally close this issue, please feel free to re-open it if you have further questions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/D-X-Y/NAS-Projects/issues/44?email_source=notifications&email_token=AIS6CU5WQRL2LERVNTZWI63Q3PZ3DA5CNFSM4KBT5X3KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH43QOY#issuecomment-570013755, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIS6CU2UPEWPCJTLSJ56LJDQ3PZ3DANCNFSM4KBT5X3A .

-- Best Regards, Renjie PI