search

automl / nasbench-1shot1

Apache License 2.0
67 stars 14 forks source link

Concat input tensor to output tensor #2

Closed ultmaster closed 4 years ago

ultmaster commented 4 years ago

https://github.com/automl/nasbench-1shot1/blob/82819476486274034ac57105e8b88cf041848c1e/optimizers/darts/model_search.py#L103

In original nasbench, input tensor is added to the concat result of selected intermediate nodes, if there is edge from input to output). But here, input tensor is concatenated.

I know if input tensor is added, the channel number would change, some of the optimizers might not work. However, it seems more reasonable to me if I just throw all architectures with edge from input to output away. Wonder why such decision is made.

Thank you.

JulienSiems commented 4 years ago

Hey! I had a look at this again and I agree that the current implementation is not as close to the NAS-Bench-101 models as it could be. Thank you for pointing this out! We will be rerunning the experiments and adjust Figure 1 to account for the changes.