search

D-X-Y / AutoDL-Projects

Automated deep learning algorithms implemented in PyTorch.
MIT License
1.56k stars 281 forks source link

Does the affine parameter of BN have effect in search phase? #39

Closed mrluin closed 4 years ago

mrluin commented 4 years ago

Thanks for your impressive work and the released code! I saw that in DARTS, BN is not learnable in the search phase. And the authors claim that

Learnable affine parameters in all batch normalizations are disabled during the search process to avoid rescaling the outputs of the candidate operations.

In contrast, BN is set to be learnable by default in the search phase of GDAS (if I didn't miss some important points). Does the affine parameter have some effect on the search phase? Could you give me some hints? Thanks in advance!

D-X-Y commented 4 years ago

Yes, "without affine" during the searching is more stable. I will update the codes soon. Thanks for your question.

mrluin commented 4 years ago

Thanks for your quick reply! Have a nice day :D