Training phase is diffrent from quark0/darts？

[Debrove]

Hi, there are some confusions

1.I try to reproduce the DARTS-V1 results by your codes, but final result is 3.28+-0.04. 2.I replaced the DARTS genotype searched by quark0/darts with DARTS_V1 and train it by your codes. Finally the result is 2.79+-0.11. It costs 35 hours. 3.I replaced the GDAS genotype with DARTS_V1 and train it. The final result is 2.72. It costs 38 hours to train.

For 1, I guess the searched architecture is not good enough though I have searched four times and pick up the best. For 2, it is lower than the result train by quark0/darts, which is 2.99+-0.04 and costs 41 hours. For 3, the reproduced GDAS result is 2.89+-0.05 (costs 54 hours to train) , which is higher than the result in 3.

So, for training phase, is any differences from quark0/darts?(I can not find any differences if I did not miss something.)

Thank you very much.

[D-X-Y]

Thanks for your interest at first. Would you mind giving me more details? Since this repo contains multiple implementations of different algorithms, I'm a little bit confused about which part are you referring to. Which script in this repo are you used for searching? Which script in this repo are you used for re-training? By "result", do you mean the error on CIFAR-10?

[Debrove]

Sorry for carelessness.

For 1, I use ./scripts-search/DARTS1V-search-NASNet-space.sh for searching and bash ./scripts/nas-infer-train.sh cifar10 DARTS_V1 96 -1 for re-training.

For 2, I use bash ./scripts/nas-infer-train.sh cifar10 DARTS_V1 96 -1 to re-train the genotype searched by quark0/darts.

For 3, I use bash ./scripts/nas-infer-train.sh cifar10 DARTS_V1 96 -1 to re-train the genotype searched by the script bash ./scripts-search/NASNet-space-search-by-GDAS.sh cifar10 1 -1.

The reason I did these is to see whether the same genotypes will have similar results on different codes.(yours and quark0s)

Thank you for your quick reply.

[D-X-Y]

I see. The re-training codes are almost the same as that in quark0/darts, but for the specific hyper-parameters, I can not remember all the details but I think I check it several months ago.

For 2, you said that my codes obtain "2.79+-0.11" but quark0/darts gets "2.99+-0.04"? Would it be a good thing, as you can obtain a better result using my codes :)

[Debrove]

One question is that the decay in configs/opts/NAS-CIFAR-V2.config and configs/search-opts/DARTS-NASNet-CIFAR.config are 0.0005, while in the paper the decay is 0.0003. Is there any suggestion?

Well, it is interesting and thank you very much.

[D-X-Y]

Nice catch! it should be my mistake

[D-X-Y]

Since the weight decay value has been corrected in the recent commit, I'm closing this issue.