ML AutoML NAS NASNet

[junxnone]

NASNet

• Learning Transferable Architectures for Scalable Image Recognition
• 基于 Neural Architecture Search with Reinforcement Learning
• Normal Cell - 输入和输出的特征图维度相同
• Reduction Cell - 输出特征图长宽减半
• NASNet Search Space
• ScheduledDropPath
• AO - RL + Policy Gradient

Arch

• Model = Normal Cell x N + Reduction Cell x M
• Normal/Reduction Cell = Block x B

  在 paper 中 B = 5

• Block = hidden states x 2 + Operation x 3

Arch	Blocks

Cell = B Block / Each Block predict 5 Steps

Step 3/4 可选 Operations	Step 5 可选 Operation / Combine two hidden states
identity	element-wise addition
1x3 then 3x1 convolution	concatenation

1x7 then 7x1 convolution 3x3 dilated convolution 3x3 average pooling 3x3 max pooling 5x5 max pooling 7x7 max pooling 1x1 convolution 3x3 convolution 3x3 depthwise-separable conv 5x5 depthwise-seperable conv 7x7 depthwise-separable conv

Search Result NASNet-A/B/C

Name	Description
NASNet-A	- B=5
NASNet-B	- B=4 - 最后没有Concatenate - Layer Normalization & Instance Normalization
NASNet-C	- B=4 - Layer Normalization & Instance Normalization

搜素到的相对比较好的 Normal Cell & Reduction Cell - NASNet-A -- |

NASNet-B	NASNet-C

Layer Normalization Instance Normalization

Reference

• paper - 2017 - Learning Transferable Architectures for Scalable Image Recognition
• TensorFlow-Slim NASNet-A Implementation/Checkpoints
• NASNet学习笔记
• 论文笔记-NAS
• 论文笔记-NASNet
• AutoMl及NAS概述：更有效地设计神经网络模型工具
• 神经网络架构搜索（NAS）综述（附AutoML资料推荐）
• 【Transferable NAS with RL】2018-CVPR-Learning Transferable Architectures for Scalable Image Recognition-论文阅读
• NASNet - pytorch

[junxnone]

junxnone/tech-io#107 junxnone/tech-io#805