search

capstone2019-neuralsearch / AC297r_2019_NAS

Harvard IACS Data Science Capstone: Neural Architecture Search (NAS) with Google
5 stars 1 forks source link

Hyperparameters #17

Closed dylanrandle closed 4 years ago

dylanrandle commented 4 years ago

Figure out which parameters to tune to make DARTS work (base dataset: MNIST/FashionMNIST)

dylanrandle commented 4 years ago

It appears that simply changing the learning rate from 0.025 to a more sensible 0.001 fixes the problem.

My conjecture is that with such a high learning rate, the model was jumping around in parameter space, always ending in local minima that do well on the training set, but only sometimes ones that generalize and perform well on validation.