ML AutoML HPO BO TPE

[junxnone]

Reference

• 2011 Algorithms for Hyper-Parameter Optimization
• 2013 Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures
• Tree-structured Parzen Estimator（TPE）
• Towards automating machine learning: benchmarking tools for hyperparameter tuning, Thorben Jensen

Brief

• TPE - Tree-structured Parzen Estimator
  • 参数空间是树形的, 存在依赖关系 - Configuration spaces are tree-structured
• 可异步并行
• TPE 是根据 p(x|y) 和 p(y) 进行建模

Steps	推导
选取 loss 阈值 y*
**
构造参数
构造函数
**
**

y*	l(x) & g(x)

Pipeline

Steps	Diagram
1 测试超参数 选择Loss 阈值 - Best - Worse
2 构建Best/Worse分布 对候选参数求分布值
3 EI = p(good)/p(bad)

[junxnone]

junxnone/tech-io#684