AutoML - NAS

[junxnone]

Reference

• paper - 2017 - Learning Transferable Architectures for Scalable Image Recognition
• paper - 2017 - Progressive Neural Architecture Search
• paper - 2018 - Efficient Neural Architecture Search via Parameter Sharing
• paper - 2019 - Neural Architecture Search: A Survey
• paper - 2019 - A Survey on Neural Architecture Search
• paper - 2020 - A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
• Awesome-NAS - D-X-Y
• Awesome-NAS-CV
• Awesome-NAS - pzhren - History
• NAS Mind Map
• 一文详解神经网络结构搜索（NAS）
• NAS Paper and Survey

Brief

• One Stage/Two Stage/OneShot
• Search Space - > Search Strategy( Architecture Optimization) -> Performance Estimation Strategy
• RL/EA -> Blocks -> GD -> One-Shot

One Stage vs Two Stage vs OneShot

NAS-Type	Description	Flow
One-Stage	搜索完之后模型可以直接使用
Two-Stage	搜索完之后要 retrain & finetune
One-Shot	search space构造成一个SuperNet share same weights

NAS Algos

Algos	Description

ENAS DARTS P-DARTS CDARTS SPOS ProxylessNAS Network Morphism TextNAS PPO | Proximal Policy Optimization Algorithms

History - Timeline

• 11/2016 - NAS & RL - Neural Architecture Search with Reinforcement Learning
• 03/2017 - Large-Scale Evolution of Image Classifiers
• 07/2017 - NASNet - Learning Transferable Architectures for Scalable Image Recognition
• 08/2017 - SMASH: One-Shot Model Architecture Search through HyperNetworks
• 12/2017 - PNAS - Progressive Neural Architecture Search
• 02/2018 - ENAS - Efficient Neural Architecture Search via Parameter Sharing
• 06/2018 - DARTS: Differentiable Architecture Search
• 12/2018 - ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

[junxnone]

107