Closed chenyaofo closed 4 years ago
Thanks for your interest! This is a very good question.
Since NASBench-101 dataset is generated using Tensorflow while our experiment settings are in pytorch, we choose to train from scratch to make sure the reimplementation causes minimum differences. However, as far as I remember, using the provided test accuracy does not change the conclusion.
If you have further discussion, please feel free to re-open this!
Why do you need to train the architectures from scratch again on NASBench-101 search space?
I find that To ensure fairness, after the search phase is completed, each method trains the top 1 architectures found by its policy from scratch to obtain ground-truth performance in D.3 in the appendix.
The test accuracy is provided in the NASBench-101 dataset, why don't you use the provided test accuracy in the Table.6 in the appendix?