Freeze-thaw bayesian optimization

[nabenabe0928]

Freeze-thaw Bayesian optimization

Main points

• Use a monotonic kernel to model learning curves
• Assume each training curve is sampled from a separate GP, but assume the time series has a correlation between one time step to another
• Use entropy search (ES) to potentially restart early-stopped configurations (, but the ES is not divided by cost.)
• Also it uses fantasizing for early-stopped configurations

Confusing points

1. Section 2 (A Kernel for Training Curves)

   • Eq. (5), $k(t, t_1) > k(t, t_2)$ if $t_1 < t_2$ (this is not a stationary kernel!!) ==> It is nice to have this property due to the monotonicity

2. Section 4

   • Anyways, it is about the computation of the posterior.
   • Apparently, the dimensionalities in the equation are a bit weird.