Optimization loop chassis

[akilandrews]

Updates include:

• Scaling training features using learned length scales
• Support for reusing optimizer instance
• Added two parameters: keep_state and probe_previous
• Update BayesianOptimization.TargetSpace.target_function with new objective function

Evaluated performance using 2D anisotropic distortion distance function (simple case) experimenting with four treatments:

• 1 (baseline) - brute force workflow following Anisotropic Metric Tutorial using single optimization step.
• 2a (opt loop) each loop iteration create new optimizer instance, initialize with new objective function, and do not probe previous points
• 2b (opt loop) each loop iteration create new optimizer instance, initialize with new objective function, and probe previous points
• 2c (opt loop) each loop iteration reuse optimizer instance, set target_function to new objective function, and do not probe previous points

[bwpriest]

Don't worry about occasional failures in the 3.X optimize checks. You probably did not do anything wrong. Occasionally the number of BO probes is insufficient to converge to a good value or the sampler produces a particularly yucky surface. I am using generous tolerances and several iterations, but sometimes there is still a failure. This is a classic problem associated with testing stochastic code.