khanrc / pt.darts

PyTorch Implementation of DARTS: Differentiable Architecture Search
MIT License
439 stars 108 forks source link

Error in deriving the final genotype #38

Open xiangzhluo opened 4 years ago

xiangzhluo commented 4 years ago

In the genotype.py (parse function), you just select the edges and the operations with the original alpha, not the alpha possibility. Sometimes it may derive wrong genotypes. For example, the alpha param for the second node is as follows: alpha = [[1, 1, 1, 1, 5, 1, 1, 1], [10, 10, 10, 11, 10, 10, 10, 10], [10, 15, 10, 10, 10, 10, 10, 10]] In your parse function, the final genotype would select the second and third edge. But larger values do not always reflect larger possibilities. Actually, we should select the first and third edge. LOL

Also, as mentioned in another issue, the formula of hessian should be as follows: hessian = [(p-n) / (2.*eps) for p, n in zip(dalpha_pos, dalpha_neg)]