search

JunrQ / NAS

Neural architecture search(NAS)
14 stars 2 forks source link

Do you have any good choice for the three levels of resource constraint? #12

Open MarkAlive opened 5 years ago

MarkAlive commented 5 years ago

It seams that 1e-9 is not suitable for your new code. Experiment outputs costs is about 6e-04, and the accompany loss is 2.414636e+00 during the former steps.

https://github.com/JunrQ/NAS/blob/f5b0f2548d7ea6e72b661b608c35b9de6afa6259/snas/snas/train_cifar10.py#L31

JunrQ commented 5 years ago

I think you can choose the weight of resource constraint as long as it's neither too big nor too small. For example, loss of resource : loss of classification = 1 : 10

That's just my point of view. Hope to help.

MarkAlive commented 5 years ago

thanks @JunrQ , these are my experiment results:

  1. resource_weight = 1e-9, loss of resource : loss of classification = 4000 : 1, valid_acc = 90%
  2. resource_weight = 5e-7, loss of resource : loss of classification = 10: 1, valid_acc = 65%