What are the differences between "Creating network architectures which encode solutions" and "addressed by neural architecture search (NAS)" ?

[onePnot1]

Hello, thank you for your excellent work!

I am really curious about this paragraph in paper.

Creating network architectures which encode solutions is a fundamentally different problem than that addressed by neural architecture search (NAS)

So could you tell me the differences between "Creating network architectures which encode solutions" and "addressed by neural architecture search (NAS)" ?

Thanks a lot.

[sujithvemi]

Let me try to take a stab at your question.

NAS is a technique to search the space of possible architectures, built from a set of building blocks (like CNN kernels, activations etc. for image data), to get the best architecture for the task at hand. This involves, training the networks for a few iterations and evaluating the improvement in performance and only keeping the best performing network(s) to train for longer.

So, as already communicated in the paper, this work is for creating network architectures that have high inductive biases for the task at hand. This means they should be able to perform the task properly right out of the box, with very minimal training of weights. Such a network is thought to be encoding the solutions in its network topology itself, than in weights. This is not the case with architectures that are found using NAS, they are not usually capable of performing the task properly without explicit weight training.

Hope I was able to answer your question. Cheers.