search

naszilla / naszilla

Naszilla is a Python library for neural architecture search (NAS)
Apache License 2.0
304 stars 46 forks source link

Path encoding in DARTS search space #12

Closed auroua closed 4 years ago

auroua commented 4 years ago

Hey @crwhite14:

The code for generating the path encoding of cell architecture in DARTS seems different from the NASBench-101. Despite the input-to-output paths, the paths that not directly connect to the output also considered to update the path encoding vector. The paths variable in the code stored all the paths; even it does not connect to the output.

https://github.com/naszilla/naszilla/blob/393e55ea97ee52d2cc79e4b290a05ad301d9401f/naszilla/nas_bench_301/cell_301.py#L220

I am confused by the code. Is this a bug, or the meaning of the input-to-output path in DARTS search space is different from the NASBench-101?

crwhite14 commented 4 years ago

Hi @auroua

Yes, this is correct. The DARTS paper describes the cell as having two input nodes, and four intermediate nodes. Additionally, there is a final output node which concatenates the output of all intermediate nodes. (This is different for the NASBench-101 paper.)

The blueprint for a cell in the original DARTS code has the variables for the nodes and edges of the four intermediate nodes. But when they build the neural architecture, the final edges to the extra output node are added in.

Therefore, to enumerate all the paths, we include all paths that terminate at an intermediate node.

auroua commented 4 years ago

Hi @crwhite14

I get it. I originally thought that only nodes that were not used as input to other nodes would be connected to output nodes.

Thanks for your reply.