Neural Architecture Search

[TreeinRandomForest]

Do this with MNIST (or a simple dataset) before the retinopathy data.

Goal: to learn an architecture that gives maximal validation score

Techniques:

1. Use the NAS module in dragonfly (https://github.com/dragonfly/dragonfly).
2. Implement the policy gradient (REINFORCE) algorithm (used in the cartpole issue) to learn an architecture. See: https://arxiv.org/abs/1611.01578 and also, https://arxiv.org/abs/1802.03268 for a much more efficient approach.

Make the following plot: x-axis: train time y-axis: validation score

Inspect the architectures learned. Use a small set of primitives (conv layers, pool layers, a few fixed kernel sizes etc.)